Statistics of Anatomic Geometry: Information Theory and Automatic Model Building

1. Statistics of Anatomic Geometry:Information Theory and Automatic Model Building Carole Twining Imaging Science and Biomedical Engineering (ISBE) University of Manchester, UK Contributions from: Rhodri Davies, Stephen Marsland, Tim Cootes, Vlad Petrovic, Roy Schestowitz, & Chris Taylor

2. Overview • Recap of Point Distribution/Statistical Shape Models PDMs/SSMs • Correspondence Problem: • Shape Representation & Correspondence • Correspondence & Statistics • Methods for establishing correspondence • Automatic Methods for Groupwise Shape Correspondence • Manipulating Correspondence not Shape • Minimum Description Length objective function • Optimisation • Extension to Images: • MDL Groupwise Registration • automatic models from unannotated image sets • Model Evaluation Criteria Slide 2

3. PCA Model PDF Shape Space Point Distribution Models (PDMs)Statistical Shape Models (SSMs) Set of Shapes & Corresponding Points Slide 3

4. Shape Space Shape & Appearance Space Adding Image Information Slide 4

5. Adding Image Information • Include image information from whole region • Correlation between shape & texture Shape & Texture Model Shape Model Slide 5

6. Active Shape & Appearance Models AAM Search ASM Search Slide 6

7. The Correspondence Problem

8. Shape Representation & Correspondence • Non-Local Representations • Fourier descriptors (e.g., SPHARM) • Medial descriptors (e.g., MREPS) • Local Representations • Point based (e.g., PDMs/SSMs) • Common Representation of training set => Correspondence • Non-local tends to give implicit correspondence • Point based gives explicit correspondence • Why does the correspondence matter? Slide 8

9. Shape Space Shape Space Correspondence & Statistics Varying correspondence varies the shape statistics Slide 9

10. Establishing Correspondence • Manual landmarking • Arbitrary parameterisations • Kelemen, Hill, Baumberg & Hogg • Shape features • Wang, Brett • Image registration • models from deformation field • Christensen, Joshi, Lavalle, Reuckert, Twining Slide 10

11. Manual Methods for Correspondence • Manual Landmarks • Interpolate for dense correspondence • May need to adjust • Problems: • Time-consuming • Subjective • Requires expert anatomical knowledge • Very difficult in 3D Slide 11

12. Arc-Length Parameterisation • Equally-space landmarks around each shape (Baumberg & Hogg) Slide 12

13. Shape Features • e.g. Curvature-based methods • Intuitive • But: • What about regions without such features? • Not really groupwise, since depends on local properties of each shape • Is it really the best correspondence? Slide 13

14. Automatic Groupwise Correspondence

15. Automatic Groupwise Correspondence Desirable features: • Groupwise: • Depends on whole set of shapes • Automatic – little or no user intervention • 2D & 3D • Runs in reasonable time! Slide 15

16. Automatic Groupwise Correspondence Optimisation Problem Framework: • Method of manipulating correspondence: • 2D & 3D • Objective function: • quantifies the ‘quality’ of the correspondence • Optimization Scheme Slide 16

17. Manipulating Correspondence

18. Shape Points Correspondence Points Manipulating Correspondence • Point-to-Point: Shape 1 Shape 2 Varying correspondence varies shape! Vary correspondence but not shape! Slide 18

19. Manipulating Correspondence • Continuous parameterisation of shape • Re-parameterising varies correspondence Slide 19

20. Sphere & Spherical Polar coordinates Shape Manipulating Correspondence • Generalises to 3D • Map surface to parameter sphere - no folds or tears • Varying parameterisation on sphere Slide 20

21. Objective Function

22. Shape Space Shape Space Objective Function • Varying Correspondence = Varying Statistics • Objective function based on model probability density function • number of model modes • compactness • quality of fit to training data • number of model parameters Slide 22

23. Shape Space MDL Objective Function • Transmit training set as encoded binary message • Shannon: • Set of possible events {i} with probabilities {pi} • Optimal codeword length for event i: -log pi • Encode whole training set of shapes: • Encoded Model: mean shape, model modes etc • Reconstruct shape space and model pdf • Each training shape: pi from model pdf • Reconstruct all training shapes • MDL Objective function = total length of message Slide 23

24. MDL Objective Function • Fit between model pdf and training data: • Probabilities for training points => better the fit, shorter the message • Too complex a model: • model parameter term large • Too few modes: • Bad fit to data & large residual • Badly chosen modes: • Bad fit to data Slide 24

25. Optimisation • Genetic algorithm search (Davies et al, 2002) • All parameters optimised simultaneously • Slow, scales badly with no of examples • More recent, multi-scale, multi-resolution approaches: • better convergence • fast enough for routine use • scales approximately linearly with no of examples (Davies et al, IPMI 2003) Slide 25

26. Results • Quantitatively better results compared to SPHARM • Differences tend to be subtle • Comparing techniques, have to go beyond visual inspection (see section on Model Evaluation Criteria) Slide 26

27. MDL Groupwise Image Registration

28. Image & Shape Correspondence • Groups of Shapes: groupwise dense correspondence • statistical models of shape variability • analysis of variation across & between populations • assist in analysing unseen examples (ASM & AAM) • Groups of Images: groupwise dense correspondence = groupwise registration • statistical models of shape & appearance • as above • MDL technique for correspondence can be applied to both (Twining et al 2005) Slide 28

29. Image Registration • Spatial Correspondence between images • Shape variation • Warp one to another • Difference is texture variation • Repeat across group => Appearance model of image set Slide 29

30. Groupwise Image Registration • MDL Objective Function • Combined shape & texture model • Define dense correspondence • triangulated points on each image & interpolate • Manipulate Correspondence • Increase resolution of mesh & repeat Slide 30

31. Results • 104 2D brain slices • Appearance Model Slide 31

32. Model Evaluation Criteria

33. Model Evaluation Criteria • Need to go beyond visual inspection, subtle differences • Generalisability: • the ability to represent unseen shapes/images which belong to the same class as those in the training set • Specificity: • the ability to only represent images similar to those seen in the training set • Quantitative comparison of models Slide 33

34. Training Set: Sample Set from model pdf: General but not Specific Specific but not General Specificity and Generalization Space of Shapes/Images Slide 34

35. :distance on image/shape space Specificity Training Set Sample Set Slide 35

36. Training Set Generalisation Ability Sample Set Slide 36

37. Objective function Specificity Generalisation Validation • Annotated/Registered Data • Perturb Registration Size of Perturbation Slide 37

38. Evaluating Brain Appearance Models Slide 38

39. Summary • Manipulating Correspondence • Shown to produce quantitatively better models • Large-scale Optimisation problem - so far, only linear models • Extension to other shape representation methods (e.g. MREPS) • Topology – manipulate parameter space: • simple, fixed topology • Multi-part objects • Differences tend to be subtle - go beyond visual inspection of results • Model evaluation criteria • Extension to groupwise image registration Slide 39

40. Questions?

41. Resources & References AAMs, ASMs • [1] T. F. Cootes, G. J. Edwards, and C. J. Taylor, Active appearance models, IEEE Trans. Pattern Anal. Machine Intell., vol. 23, no. 6, pp. 681-685, 2001. • [2] T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham, Active shape models – their training and application, Computer Vision and Image Understanding, 61(1), 38-59, 1995 • [3] T. F. Cootes, A. Hill, C. J. Taylor, and J. Haslam, The use of active shape models for locating structures in medical images, Image and Vision Computing, vol. 12, no. 6, pp. 276-285, July 1994. • [4] B. van Ginneken, A.F.Frangi, J.J.Stall, and B. ter Haar Romeny, Active shape model segmentation with optimal features, IEEE Trans. Med. Imag., vol. 21, pp. 924-933, 2002. • [5] P. Smyth, C. Taylor, and J. Adams, Vertebral shape: Automatic measurement with active shape models, Radiology, vol. 211, no. 2, pp. 571-578, 1999. • [6] N. Duta and M. Sonka, Segmentation and interpretation of MR brain images: An improved active shape model, IEEE Trans. Med. Imag., vol. 17, pp. 1049-1067, 1998. Further references, as well as notes on the historical meanderings in the development of these techniques can be found on Tim Cootes’ website: http://www.isbe.man.ac.uk/~bim/ Slide 41

42. Resources & References MREPS • [7] S. M. Pizer, D. Eberly, D. S. Fritsch, and B. S. Morse, Zoom-invariant vision of figural shape: The mathematics of cores, Computer Vision and Image Understanding, vol. 69, no. 1, pp. 055-071, 1998. Fourier descriptors, spherical harmonics & SPHARM • [8] C. Brechb¨uhler, G. Gerig, and O. Kubler, Parameterisation of closed surfaces for 3D shape description, Computer Vision, Graphics and Image Processing, vol. 61, pp. 154-170, 1995. • [9] A. Kelemen, G. Szekely, and G. Gerig, Elastic model-based segmentation of 3D neurological data sets, IEEE Trans. Med. Imag., vol. 18, no. 10, pp. 828-839, Oct. 1999. • [10] C. Brechb¨uhler, G. Gerig, and O. K uhler, Parametrization of closed surfaces for 3D shape description, Computer Vision and Image Understanding, vol. 61, no. 2, pp. 154-170, 1995. • [11] G. Szekely, A. Kelemen, C. Brechbuhler, and G. Gerig, Segmentation of 2-D and 3-D objects from MRI volume data using constrained elastic deformations of flexible fourier contour and surface models, Medical Image Analysis, vol. 1, pp. 19-34, 1996. Slide 42

43. Resources & References Fourier descriptors, spherical harmonics & SPHARM • [12] D. Meier and E. Fisher, Parameter space warping: Shape-based correspondence between morphologically different objects, IEEE Trans. Med. Imag., vol. 21, no. 1, pp. 31-47, Jan. 2002. • [13] M. Styner, J. Liberman, and G. Gerig, Boundary and medial shape analysis of the hippocampus in schizophrenia, in Proc. International Conference on Medical Image Computing and Computer Aided Intervention (MICCAI), 2003, pp. 464-471. Feature-Based Shape correspondence • [14] A. D. Brett, A. Hill, and C. J. Taylor, A method of automatic landmark generation for automated 3D PDM construction, Image and Vision Computing, vol. 18, pp. 739-748, 2000. • [15] Y. Wang, B. S. Peterson, and L. H. Staib, Shape-based 3D surface correspondence using geodesics and local geometry, in Proc. IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2000, pp. 644-651. • [16] G. Subsol, J. Thirion, and N. Ayache, A scheme for automatically building three-dimensional morphometric anatomical atlases: application to a skull atlas, Medical Image Analysis, vol. 2, no. 1, pp. 37-60, 1998. Slide 43

44. Resources & References Elastic and Distortion based methods of shape correspondence • [17] M. Kaus, V. Pekar, C. Lorenz, R. Truyen, S. Lobregt, and J. Weese, Automated 3-D PDM construction from segmented images using deformable models, IEEE Trans. Med. Imag., vol. 22, no. 8, pp. 1005-1013, Aug. 2003. • [18] C. Shelton, Morphable surface models, International Journal of Computer Vision, vol. 38, pp. 75-91, 2000. • [19] S. Sclaroff and A. P. Pentland, Modal matching for correspondence and recognition, IEEE Trans. Pattern Anal. Machine Intell., vol. 17, no. 6, pp. 545-561, 1995. • [20] F. L. Bookstein, Landmark methods for forms without landmarks: morphometrics of group differences in outline shape, Medical Image Analysis, vol. 1, no. 3, pp. 225-244, 1997. Minimum Description Length This is the information theory stuff behind MDL. • [21] J. Rissanen, Lectures on Statistical Modeling Theory, http:\\www.cs.tut.fi\~rissanen\papers\lectures.pdf • [22] J. Rissanen, Stochastic Complexity in Statistical Inquiry, World Scientific Press, 1989. Slide 44

45. Resources & References MDL for Shape Correspondence Approximate MDL Note that the freely available code distributed by Thodberg is only approximate MDL, not full state-ofthe- art MDL as used by other groups. In fact, the objective function used in these papers is equivalent to what is used to initialise other algorithms. This fact has caused a little confusion in the literature. • [23] H. Thodberg, MDL shape and appearance models, in Proc. 18th Conference on Information Processing in Medical Imaging (IPMI), 2003, pp. 51-62. • [24] H. Thodberg and H. Olafsdottir, Adding curvature to MDL shape models, in Proc. 14th British Machine Vision Conference (BMVC), vol. 2, 2003, pp. 251-260. • [25] T. Heimann, I. Wolf, T. G. Williams, and H.-P. Meinzer, 3D Active Shape Models Using Gradient Descent Optimization of Description Length , IPMI 2005. MDL for 2D Shape This uses the initial genetic algorithm search, which was later improved upon. • [26] R. H. Davies, C. J. Twining, T. F. Cootes, J. C. Waterton, and C. J. Taylor, A minimum description length approach to statistical shape modelling, IEEE Trans. Med. Imag., vol. 21, no. 5, pp. 525-537, May 2002. • [27] R. H. Davies, C. J. Twining, P. D. Allen, T. F. Cootes, and C. J. Taylor, Building optimal 2D statistical shape models, Image and Vision Computing, vol. 21, pp. 1171-1182, 2003. Slide 45

46. Resources & References MDL for 3D Shape • [28] R. H. Davies, C. J. Twining, T. F. Cootes, J. C. Waterton, and C. J. Taylor, 3D statistical shape models using direct optimisation of description length, in Proc. 7th European Conference on Computer Vision (ECCV), 2002, pp. 3-21. MDL for Image Registration • [29] C. J. Twining, T. Cootes, S. Marsland, V. Petrovic, R. Schestowitz, and C. J. Taylor, A Unified Information-Theoretic Approach to Groupwise Non-Rigid Registration and Model Building, Presented at IPMI 2005 • [30] C. J. Twining, S. Marsland, and C. J. Taylor, Groupwise Non-Rigid Registration: The Minimum Description Length Approach, In Proceedings of BMVC 2004. • [31] C.J. Twining and S. Marsland, A Unified Information-Theoretic Approach to the Correspondence Problem in Image Registration, International Conference on Pattern Recognition (ICPR), Cambridge, U.K. 2004. Slide 46