Dynamic Factor Analysis

1. Dynamic Factor Analysis Ellen L. Hamaker Methods and Statistics Faculty of Social Sciences Utrecht University The Netherlands

2. Outline • Introduction • Time series analysis • Linear Kalman filter • Illustration 1 • Regime-switching Kalman filter • Illustration 2 • Discussion

3. 2 kinds of statistical techniques Concerning means of populations t-test ANOVA MANOVA Concerning covariance structure of populations correlation regression analysis factor analysis path analysis Means and covariance structures combined in SEM

4. How did it start? In 1884 Galton established his anthropometric laboratory and measured mental faculties and physical appearances of 9000 visitors. His research subject was: variation in the population. Galton believed most mental and physical features were inherited. He was worried that the protection of the weak (i.e., the poor) would interfere with the mechanisms of natural selection. Galton is the founder of eugenics.

5. Other important eugenicists Pearson follower of Galton, and inventor of the product-moment correlation coefficient Spearman student of Wundt, and inventor of factor analysis, and the concept of general intelligence Fisher mathematician, and inventor of: ANOVA, experimental designs, principle of maximum likelihood, inferential statistics, null-hypothesis testing, F-test, Fisher information, non-parametric statistics, et cetera, et cetera…

6. Mathematical statistics The statistical techniques used in the social sciences were developed to study heredity. Hence, they have two important features: a. heredity operates at level of population: same holds for these techniques b. biometrics is concerned with studying trait- like variables, not processes

7. What is the problem? Our standard techniques focus on characteristics of the population (means, correlations, proportions). BUT… results are not always generalizable to the individual. For instance: • if we find a beneficial effect of therapy at the group level, this does not guarantee that every individual improved • if we find a smooth change at the group level, it is possible that at the individual level there is a sudden change • if 20% of clients are cured after treatment, this does not imply that an individual has a 20% change of being cured

8. E.g., correlation interindividual intraindividual mistakes mistakes words per minute words per minute

9. Who makes this mistake? Personality processes, by definition, involve some change in thoughts, feelings and actions of an individual; all these intra-individual changes seem to be mirrored by interindividual differences in characteristic ways of thinking, feeling and acting. McCrae & John (1992) shy sociable

10. The same in formulas Let i be the subject index, and x and y be two variables. INTRAindividual correlation: INTERindividual correlation

11. Questions about processes Is the relationship at the INTRAindividual level identical to the relationship at the INTERindividual level? If not, is there an universal relationship? If not, can the differences between individuals with respect to their dynamics be related to other individual differences?

12. Outline • Introduction • Time series analysis • Linear Kalman filter • Illustration 1 • Regime-switching Kalman filter • Illustration 2 • Discussion

13. Dynamic system A DS is a set of equations that describe how the state of the system changes as a function of its previous state. Characteristics of a DS: • 1 or more variables • state = values of the variables • stochastic/deterministic • discrete or continuous time • linear or nonlinear Time series analysis is a technique to study uni- or multivariate, stochastic systems in discrete time, which may be linear or nonlinear.

14. Autoregressive models at-2 at-1 at at+1 at+2 yt-2 yt-1 yt yt+1 yt+2 y*t-2 y*t-1 y*t y*t+1 y*t+2 a*t-2 a*t-1 a*t a*t+1 a*t+2

15. Time series Unrelated series: first series contains autocorrelation second series is white noise Two related series: first contains positive autocorrelation second contains negative autocorrelation

16. Dynamic factor model A DFM relates multiple indicators to 1 or more latent variables (factor model). Because the variables are measured repeatedly (T>50), the dynamics can be modeled (i.e., the structure in the changes over time). Two ways of including lagged relationships: • lagged factor loadings • latent VARMA process

17. DFM with lagged factor loadings ft-2 ft-1 ft ft+1 yt-2 yt-2 yt-2 yt-2 yt-1 yt-1 yt-1 yt-1 yt yt yt yt yt+1 yt+1 yt+1 yt+1

18. DFM with latent VARMA process at-1 at-1 at at+1 ft-2 ft-1 ft ft+1 yt-2 yt-2 yt-2 yt-2 yt-1 yt-1 yt-1 yt-1 yt yt yt yt yt+1 yt+1 yt+1 yt+1

19. Outline • Introduction • Time series analysis • Linear Kalman filter • Illustration 1 • Regime-switching Kalman filter • Illustration 2 • Discussion

20. Kalman filter The Kalman filter is an algorithm for estimating the latent states, and for predicting time series models. It requires the model to be reformulated in state-space format, i.e.:

21. Goal of Kalman filter Obtain estimates for the states at (and predict future observations). t = T ? t = T ?

22. Estimation of model parameters t = T ?

23. Outline • Introduction • Time series analysis • Linear Kalman filter • Illustration 1 • Regime-switching Kalman filter • Illustration 2: nonlinear KF extension • Discussion

24. Daily measures of E & N Data: 90 repeated measures in 22 subjects of states associated with the Five Factor Model of personality. Neuroticism items Extraversion items total variance state variance trait variance

25. Results 1. Does every one have the same 2-factor structure? - 3 persons out of 22 not - only small groups with same factor loadings 2. Are there similarties in dynamics? + Et-1 Et - at-1 at - - + ut-1 ut + Nt-1 Nt

26. Outline • Introduction • Time series analysis • Linear Kalman filter • Illustration 1 • Regime-switching Kalman filter • Illustration 2 • Discussion

27. State-space model with regime-switching Regimes can be thought of as states that differ from each other with respect to their parameters. where St is an unobserved discrete-valued Markov chain.

28. Markov-switching process Let’s focus on a 2-regimes first-order Markov-switching process. Thus we have: St = 1,2. For each regime there is a probability of staying in the same regime, and a probability of switching to the other regime.

29. KF with Markov-switching Because we do not know in which regime the process is at any occasion, we have to estimate all possibilities. Hence, we get 4 (M*M) predictions and 4 updates:

30. Collapsing the posteriors This implies that at each step we get an M-fold increase in cases (2,4,8,16,32,…). To overcome this problem, the M2 updates are reduced to M updates through: Hence, to collapse the M2 posteriors in M posteriors, we need the probabilities Pr[St-1 = i|St = j, Yt]. These are obtained with the Hamilton filter.

31. Hamilton filter of the probabilities

32. Outline • Introduction • Time series analysis • Linear Kalman filter • Illustration 1 • Regime-switching Kalman filter • Illustration 2 • Discussion

33. Positive and negative affect Daily measurements with palm handheld using the PANAS. Question: Are there distinct regimes in daily affect fluctuations? Positive affect Negative affect

34. Negative affect subject 10 Linear model: AIC: 108.52 BIC: 115.95 Two regime model: AIC: 72.32 BIC: 92.05

35. Negative affect subject 5 Linear model: AIC: 80.79 BIC: 88.35 Two regime model: AIC: 69.04 BIC: 89.21

36. Outline • Introduction • Time series analysis • Linear Kalman filter • Illustration 1 • Regime-switching Kalman filter • Illustration 2 • Discussion

37. Conclusion Today we looked at models for: • multiple indicators • multiple subjects • regime switching TSA allows us to model processes where they take place: at the level of the individual. There are different ways in which we can combine information obtained from multiple subjects.

38. Ain’t seen nothing yet! Other possibilities: • transition probabilities as functions of observed variables • smoothly changing parameters • deterministic trends and cycles (weekly, monthly) • difference scores • intervention analysis • change-point models • threshold models • ordinal data • include predictors (situational features) • include a partner (spouses, therapist-client, mother-child) • and much much more…

39. Thank youemail: e.l.hamaker@uu.nl