1 / 20

Causal Models as Minimal Descriptions of Multivariate Systems

Causal Models as Minimal Descriptions of Multivariate Systems. Jan Lemeire June 15 th 2006. What can be learnt about the world from observations?. We have to look for regularities & model them. MDL-approach to Learning. Occam’s Razor “ Among equivalent models c hoose the simplest one .”

Download Presentation

Causal Models as Minimal Descriptions of Multivariate Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Causal Models as Minimal Descriptions of Multivariate Systems Jan Lemeire June 15th 2006 Causality & MDL

  2. What can be learnt about the world from observations? • We have to look for regularities • & model them Causality & MDL

  3. MDL-approach to Learning • Occam’s Razor “Among equivalent models choose the simplest one.” • Minimum Description Length (MDL) “Select model that describes data with minimal #bits.” model = shortest program that outputs data length of program = Kolmogorov Complexity Learning = finding regularities = compression Causality & MDL

  4. Randomness vs. Regularity • 0110001101011010101 random string=incompressible=maximal information • 010101010101010101 regularity of repetitionallows compression Separation by the Two-part code Causality & MDL

  5. Model of Multivariate Systems • Variables • Experimental data Probabilistic model of joint distribution with minimal description length? Causality & MDL

  6. 1 variable • Average code length = Shannon entropy of P(x) • Multiple variables • With help of other, P(E| A…D) (CPD) • Factorization • Mutual information decreases entropy of variable Causality & MDL

  7. I. Conditional Independencies • Reduction of factorization complexity • Bayesian Network Ordering 1 Ordering 2 Causality & MDL

  8. II. Faithfulness Joint Distribution Directed Acyclic Graph Conditional independencies  d-separation Theorem: if a faithful graph exists, it is the minimal factorization. Causality & MDL

  9. III. Causal Interpretation • Definition through interventions Causality & MDL

  10. Reductionism • Causality = reductionism • Canonical representation: unique, minimal, independent • Building block = P(Xi|parentsi) • Whole theory is based on modularity like asymmetry of causality • Intervention • = change of block Causality & MDL

  11. Ultimate motivation for causality Model = canonical representation able to explain all regularities • close to reality Reality Learnt Example taken from Spirtes, Glymour and Scheines 1993, Fig. 3-23 Causality & MDL

  12. Causal model is MDL of joint distribution if Incompressible Incompressible (random distribution) Causality & MDL

  13. A Bayesian network with unrelated, random CPDs is faithful • d-separation tells what we can expect from a causal model • Eg. D depends on C, unless a dependency in P(D|C,E) P(d1|c0,e0).P(e0)+ P(d1|c0,e1).P(e1) = P(d1|c1,e0).P(e0)+ P(d1|c1,e1).P(e1) Causality & MDL

  14. When do causal models become incorrect? • Other regularities! Causality & MDL

  15. A. Lower-level regularities • Compression of the distributions Causality & MDL

  16. B. Better description form • Pattern • in figure random patterns -> distribution Causal model?? • Other models are better • Why? Complete symmetry among the variables Causality & MDL

  17. C. Interference with independencies X and Y independent by cancellation of X→U → Y and X → V → Y • dependency of both paths • = regularity Causality & MDL

  18. Violation of weak transitivity condition One of the necessary conditions for faithfulness Causality & MDL

  19. Deterministic relations • Y=f(X1, X2) • Y becomes (unexpectedly) independent from Z conditioned on X1 and X2 • ~ violation of the intersection condition Solution: augmented model - add regularity to model - adapt inference algorithms • Learning algorithm: • variables possibly contain equivalent information about another • Choose simplest relation Causality & MDL

  20. Conclusions • Interpretation of causality by the regularities • Canonical, faithful representation • ‘Describe all regularities’ • Causality is just one type of regularity? • Occam’s Razor works • Choice of simplest model • models close to ‘reality’ • but what is reality? • Atomic description of regularities that we observe? Papers, references and demos: http://parallel.vub.ac.be Causality & MDL

More Related