Groups Change Too: Analyzing Repeated Measures on Individuals Embedded Within Dynamic Groups

1. Groups Change Too: Analyzing Repeated Measures on Individuals Embedded Within Dynamic Groups Daniel J. Bauer

2. Outline of Talk • Goal: • To offer a more realistic model for repeated measures data when individuals are clustered within groups that undergo structural or functional change over time • Roadmap: • Causes and consequences of clustered data • Multilevel modeling • Analyzing change over time • Stable versus dynamic groups • Two applications of dynamic group models

3. Clustering is a Natural Feature of Data Humans exist within a social ecology including both natural and constructed groups (e.g., family and school)

4. Clustering Usually Implies Correlation • Observations on individuals from the same group tend to be correlated • Peer groups subject to selection effects (homophily) and socialization effects (group norms) • Schools include students drawn from similar sociodemographic backgrounds, and students are exposed to common teachers and curricula • Family members have common genes, environmental exposures, and social influences • Yet most statistical models assume independence of observations (more specifically, independent residuals)

5. Consequences of Ignoring Dependence • What happens if we erroneously analyze the data as if they were independent? • Standard errors, test statistics, degrees of freedom, p-values, and confidence intervals are all incorrect • Tests tend to be too liberal, inflating Type I errors • Most importantly, we neglect important processes in the data • How similar are individuals within groups? • How strong are group effects on individuals? • What predictors account for within- versus between-group differences?

6. Appropriately Analyzing Clustered Data • There are several possible ways to analyze cluster-correlated data • Fixed-effects approaches • Generalized estimating equations • Multilevel models with random effects • Multilevel models offer unique insights into both individual and group-level processes

7. A Classic Example • Science achievement scores for student from different schools:

8. A Simple Multilevel Model • A basic two-level model for clustered data: Variance component associated with each random effect Overall average (fixed effect) Group-level influences (random effect) Correlation between individuals’ scores Individual-level influences that are independent of group (random effect)

9. The Variance Components • Here we see how each component of variability maps onto our plot of the data b0 vj rij

10. Extending the Model • Normally, our next step would be to incorporate predictors at the individual and group level to explain each source of variability in the data • Suppose, however, we didn’t just measure our outcome once, but multiply over time, with the goal of capturing individual trajectories of change

11. Modeling Individual Trajectories y Personi=1 in Group j=1 0 1 2 3 Time

12. Modeling Individual Trajectories Person 1, Group 1 y Person 2, Group 1 Mean Person 3, Group 2 Person 4, Group 2 Mean Trajectory Individual Differences Time-Specific Residual Group Effect 0 1 2 3 Time

13. Taking a Closer Look • This is a typical three-level model for capturing individual change when individuals are clustered within groups • Note that the group effect, vj, is constant over time • Is this consistent with theory? Mean Trajectory Individual Differences Time-Specific Residual Group Effect

14. Dynamic Groups Just an individuals change, so too does the social ecology Chronosystem

15. Dynamic Groups • We refer to dynamic groups as those that undergo structural and/or functional change over time yet maintain their core integrity as units • Examples: • Rockbridge and Hickman High Schools both experience turnover in students, teachers, administrators, and curricula, yet continue to be characterized by distinctive school cultures • The Jones Family undergoes structural changes as a consequence of divorce, remarriage, and child birth, and undergoes functional changes as a consequence of parental addiction and unemployment, yet the Jones Family remains distinct from the Smith Family

16. Rewriting the Model • With dynamic groups, we would expect group effects to be correlated over time, but not necessarily constant • A more realistic model might thus be • The group effect is now time-varying • Key is then to discern its temporal structure Mean Trajectory Individual Differences Time-Specific Residual Group Effect

17. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? ?

18. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? Traditional “stable groups” model Correlated 1.0 over time

19. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? Toeplitzmodel Banded Covariance Matrix

20. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? Stabilization model Stabilizing Banded Covariance Matrix

21. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? Compound symmetric model Equal covariances

22. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? AR(1) model Exponential Decay

23. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? ARMA(1,1) model Rapid Decay

24. Temporal Structure of Group Effects • Say we have 4 time points • How should we structure the covariance matrix of the group effects over time? Unstructured model No Structure Imposed

25. Why It Matters • Specifying a poor temporal structure for the group effects risks • Incorrect tests of regression coefficients for predictors • Biased estimates of variance components at each level of the model • Occluding important findings regarding the nature and stability of group effects over time • Goal is thus to identify a theoretically plausible structure that fits the data well • Some structures are nested and can be compared using LRT • Others are non-nested and can be compared by BIC, AIC, etc.

26. Example: Attitudes About Science • Data drawn from the Longitudinal Study of American Youth (LSAY) • Cohort 1 (N=2091): 1987, 1988, 1989 • Cohort 2 (N=1407): 1990, 1991, 1992 • 51 schools, 3498 students, 7756 observations from grades 10-12 • Outcome is an IRT developmental scale score of science ability:

27. Goals for analysis • Evaluate the relationship between religious attitudes towards science and science achievement • “Science undermines morality” • “We need less science, more faith” • “The theory of evolution is true” (R) • “The bible is God’s word” • Separate within-school and between-school effects, while controlling for SES • Obtain accurate tests of these effects by appropriately accounting for school effects • Determine the temporal structure of school effects

28. Fitted Model • Captures average trajectory for each cohort • Captures within- and between- school effects of SES and attitudes • Captures individual differences in change over time, time-varying school effects, and residuals from the individual trajectories

29. Selecting a Temporal Structure Traditional models Dynamic group models

30. Fixed Effects * p<.05

31. School Effects Over Time 87 88 89 90 91 92 87 88 89 90 91 92 Traditional model with random intercept for school assumes constant school effect over time 87 88 89 90 91 92 87 88 89 90 91 92 Dynamic group model shows diminishing correlation of school effect over time ** Superior model fit

32. Summary • Though the regression coefficient estimates are similar, the dynamic groups model fits the data better and likely provides more accurate tests of these coefficients • Suggests both within and between-school effects of fundamentalist religious attitudes on science achievement • The dynamic groups model also provides insights into the stability and change of school effects over time • School effects highly stable from one year to the next • But the correlation decays to .62 over a period of 4 years, indicating some drift in nature of school effects over time

33. Example: Family Effects on Psychopathology • Data drawn from the Michigan Longitudinal Study (PI: Zucker) • 280 families, 588 children, 2468 repeated measures • Repeated measures included from age 11-17 and span 12 calendar years • Outcomes are IRT scores of self-reported externalizing and depression • Primary goal is to examine temporal stability of family effects on psychopathology • Ancillary goals are to estimate trajectories of externalizing and depression for boys and girls, and to evaluate added risk due to parental impairment (alcoholism, depression, ASP)

34. Fitted Model • Captures differences in average trajectories of girls and boys • Captures effects of parental impairment • Captures individual differences in change over time, time-varying family effects, and residuals from the individual trajectories

35. Selecting a Temporal Structure • For both outcomes, the AR(1) dynamic group model fits best

36. Fixed Effects * p<.05

37. Family Effects Over Time

38. Summary • Gender differences consistent with other literature • Parental history of alcoholism elevates risk of both depression and externalizing, and this is compounded by history of ASP • History of parental depression does not have a significant effect • Family effects are highly fluid • Afamily that is troubled in one year is likely to continue to function poorly in the next year or two, but may right itself over the longer term • Conversely, a family functioning well at one point in time is not immune from later difficulties • Family effects less stable for externalizing than depression

39. Conclusions • Standard multilevel models fail to account for the fact that groups undergo change over time • The effect of the group on its members is unlikely to be constant • We propose the use of dynamic group models to obtain new insights on the temporal structure of group effects • At a time lag of five years, school effects on science achievement were correlated .62 • In contrast, family effects were correlated .53 for depression and only .25 for externalizing behavior • This difference in stability is perhaps not surprising. Schools are large institutions with a great deal of inertia, whereas families are small groups that are potentially more vulnerable to stochastic events

40. Acknowledgements and Disclaimer The project described was supported by supported by National Institutes of Health grants R01 DA 025198 (PI: Antonio Morgan-Lopez), R37 AA 07065 (PI: Robert Zucker), and R01 DA 015398 (PIs: Andrea Hussong and Patrick Curran). The content is solely the responsibility of the author and does not represent the official views of the National Institute on Drug Abuse, National Institute on Alcohol Abuse and Alcoholism, or the National Institutes of Health. NishaGottfredson Danielle Dean Robert Zucker Antonio Morgan-Lopez Andrea Hussong