Medical Simulation

1. Medical Simulation Talk by Lisa Lyons

2. Surgery Simulation Requirements • Realistic visualization of internal organs • Organs react realistically in real time to: • User interactions • Environmental restrictions • Organs react to typical surgeon’s gestures through geometric and topological modifications

3. Surgery Simulation Grouping • First generation: • Only deal with geometric nature of human anatomy • Second generation: • + permit physical interactions with anatomy • Include needle-type, exploration-type, catheter installation-type simulators as well as simulators that permit training in only one task and full simulators • Third generation: • + consider functional nature of organs

4. Outline • Physical Modeling • Reduction of Computing Time • Collision Detection • Example Systems • Results and Conclusion

5. Anatomical Model of the Liver • Data set consists of about 180 slices of frozen human tissue that has been put through CT scan • Enhance contrast • Apply edge detection • Semi-automatic deformable models → binary images • Stack images to form 3D binary image [Montagnat, 1997]

6. Simplex Meshes • Better than marching cubes – avoids “staircase effects • Developed by Delingette to represent 3D objects [Delingette, 1994] • Adaptable (figure to right) • Working on a method to extract liver models from CT images

7. Force Feedback • How physically realistic the model is correlated with how realistic force feedback is • Model deforms with surgeon’s motion • Contact force may be computed from deformation • Force generated back to surgeon through mechanical actuators

8. Method uses linear elasticity as an approximation for tissue deformation • Let the configuration of an elastic body be defined as Ω • A field of volumetric and surface forces f acts on the body so it has a new configuration Ω* • We want the displacement field u which associates the initial configuration of any particle with its final configuration • Use FEM – Lagrange elements of type P1 [Bathe, 1996] • Formulate the problem as a linear system • Where [K] is the 3n by 3n stiffness matrix and n is the number of mesh vertices (more on this in a minute)

9. Only thing we know is endoscope position • must use displacement not force constraints • Given some displacements between the surgical tool and the body, we can find • Force on end effecter • Global deformation • Now we use variational formulation and Lagrange multipliers to minimize • Include constraints u = u* • Solving for λi gives the opposite of the necessary forces to impose the displacement u* • See Appendix A [Cotin, 1999] for full derivation

10. Stiffness matrix containing 3X3 “mini-matrix” of stiffness information for each node Matrix composed of a 3X3 identity matrix for each constrained segment (k) Forces required to obtain desired state Desired displacements of k nodes

11. Linear Representation • In theory, this behavior is only physically correct for small displacements • Force feedback limits the range of deformations • Feedback force on surgeon’s hand will increase as deformation increases

12. Quasi Non-Linear Representation • Mix of linear representation and empirical results using a cylindrical piece of brain tissue • [Chinsei, 1997] found that deformation depends on loading speed and is nonlinear

13. Outline • Physical Modeling • Reduction of Computing Time • Collision Detection • Example Systems • Results and Conclusion

14. Computation Time • Number of mesh vertices has high impact • Makes matrices larger • Must use speedups • Cannot make necessary calculations in real-time

15. Pre-Computation Algorithm • Specify a set of nodes to remain fixed • Don’t have to set all three dof • For every “free” node k and degree of freedom on the surface, emplace an “elementary” displacement constraint (δ) • Denote this as • Compute the displacement of every free node n in the mesh with respect to every node k • Store as set of 3X3 tensors • Compute elementary force at each constrained node k • Store as 3X3 tensors

16. Solving The Linear System • Must be solved 3m times where m is the total number of free nodes inside the tetrahedral mesh • Can take anywhere from a few minutes to several hours

17. Linear Elasticity • For any n where k≠ n, the relation between n and k is • Superposition may be used to find the total displacement of a node but some modifications must be made

18. Use tensors of deformation found in preprocessing to generate a vector of modified constraints where and

19. From this, we can find the displacement of any node • The force that must be applied to each node k to produce these displacements is

20. Quasi Non-Linear Elastic Deformations • Computing times for a realistic looking liver model:

21. Outline • Physical Modeling • Reduction of Computing Time • Collision Detection • Example Systems • Results and Conclusion

22. Collision Detection • Work discussed so far uses bounding boxes with a hash table • We know about these so lets move on to a new problem – simulating the folds of the intestines

23. Simulating Intestines • Goal is simulator to allow doctors to practice a surgery that involves pulling and folding the intestines [Raghupathi L. et. al., 2003] • Real problem here is self-collsions • Complicated by tissue called mesentery • Connects small intestine and blood vessels

24. Model • Resting position: • Intestines look like folded curves lying in a cylinder • Mesentery is defined as line segments connecting folded intestine to the axis of the cylinder • Mechanical model uses masses and springs

25. Collisions Between Intestines • Model intestines like cylinders • Find distance between their principle axes • “Active pairs” • Local minima satisfying certain distance threshold • Updated every time step • N additional random pairs of segments also generated every time step • These are tested and thrown out if they are over the threshold or already represent a minimal pair

26. Mesentery Collisions • Complexity would be too high for real-time without approximation • Don’t consider mesentery-mesentery interactions • Adaptive convergence • Replace segment S1 by closest neighbor S to S2 and then replace S2 with neighbor closest to S • When collision occurs, recursive search begins across neighbors

27. Results and Demo Video

28. Outline • Physical Modeling • Reduction of Computing Time • Collision Detection • Example Systems • Results and Conclusion

29. GeRTiSS • The Generic Real Time Surgery Simulator [Monserrat et al., 2003]

30. Scene Generator • Allows user to select tools and organs needed • Systems contains modeling parameters for a variety of organs • Mass-spring model • Boundary element based model (BEM)

31. Scene Generator • Tools: • Loading organs • Establishing input points for instruments • Associating different physical properties with organs • Establishing boundary conditions • Linking tissues • Adding special tissues • Associating textures to organs

32. Surgery Simulator • Takes a scene and allows user to train • User can have interaction with organs: • Cut • Cauterize • Drag • Clip • User can exchange instruments • User is assessed at the end based on how many incorrect actions were taken

33. Results • Use 450 MHz Pentium III with 256 MB memory • Computational Costs:

34. User Interface for Laproscopy

35. Haptics • For good visual image 15Hz refresh rate • For good haptic stimulus 500 Hz refresh rate • Use a PC cluster to solve this • Cost of force feedback devices makes simulator 4X more expensive than without

36. Cataract Surgery Simulation • Surgery aims to extract cataract and replace it with intraocular lens [Agus et al., 2006] • Training is important • Simulation allows: • Flexibility • Gradual increase in difficulty • Exposure to rare events • Quantification of performance

37. The Procedure • Phacoemulsification: breaking hardened lens into fragments and removing them with a small sucker using the phacoemulsificator • Create z-shaped corneal tunnel • Capsulorhexis: removing the anterior capsule to uncover the upper surface of the crystalline

38. Methods • Decoupled simulation: • Fast subsystem for surgical instrument tracking and slower one for visual feedback • Slow subsystem does global simulation and interaction of devices and eye • Slow subsystem can be further broken into individual visual effects • Force feedback is useless in this surgery • Must use eye globe visualization • Conjugate gradient to minimize energy constraints gives equilibrium position • Rotate to reduce deformation

39. Capsulorhexis Simulation • Use triangular mesh with a mass-spring network mapped over it • Mass particles may be anchored, scripted or free • Gravity, viscosity and springs contribute to acceleration • Weak springs simulate sticking effects • Solve ODE using semi FSAL (First Same as Last) • Velocity found using implicit method and feedback on position is computed explicitly • Correction routine applied after each step to correct position and velocity as required by constraints • Tearing – breaking overextended springs

40. Phacoemulsification Simulation • Lens as collection of simplices • Tetrahedron mesh with particles placed at barycenters • Links connecting particles maintained for rendering and determining independent particles • Photoemulsificator modeled by eroding particles in a zone of influence • Employ Russian roulette scheme to decide which particles to erode • When particles are removed, simplicial mesh is updated • Idea is to replace energies by geometric constraints and forces by distance from current position to goal • Each connected subset of points is associated with a point cloud • Shape matching with undeformed rest state to determine goal positions

41. Results

42. Outline • Physical Modeling • Reduction of Computing Time • Collision Detection • Example Systems • Results and Conclusion

43. Surgical device with force feedback simulation Visual feedback

44. Where is the future?

45. Appendix A – Collision Response • Tried penalty and constraint methods but stability of the system was reduced • Instead alter displacement velocities to avoid penetration

46. Appendix A (cont.) • Interpolating: • Need force f’ = f so we have: • New velocities are: • Substituting we get:

47. Appendix A (cont) • Solving for f gives: • Condition for avoiding penetration takes radii into account: • The force required to change the positions of the endpoints to satisfy these conditions is:

48. References • Marco Agus, Enrico Gobbetti, Giovanni Pintore, Gianluigi Zanetti, and Antonio Zorcolo. Real-time Cataract Surgery Simulation for Training. In Eurographics Italian Chapter Conference. Eurographics Association, 2006. • K.-J. Bathe, Finite Element Procedures. Prentice Hall, 1996. • K. Chinsei and K. Miller, “Compression of Swine Brain Tissue Experiment In Vitro,” J. Mechanical Eng. Laboratory, pp. 106-115, 1997. • S. Cotin, H. Delingette, and N. Ayache. “A Hybrid Elastic Model allowing Real-Time Cutting, Deformations and Force-Feedback for Surgery Training and Simulation.” The Visual Computer, 16(8):437-452, 2000. • Cotin, S.; Delingette, H.; Ayache, N., "Real-time elastic deformations of soft tissues for surgery simulation," Visualization and Computer Graphics, IEEE Transactions on , vol.5, no.1, pp.62-73, Jan-Mar 1999 • H. Delingette, ”Simplex Meshes: A General Representation for 3D Shape Reconstruction,” Technical Report 2214, INRIA, Mar. 1994. • Y.C. Fung, Biomechanics-Mechanical Properties of Living Tissues, second ed. Springer-Verlag, 1993. • Carlos Monserrat, Oscar López, Ullrich Meier, Mariano Alcañiz Raya, M. Carmen Juan Lizandra, Vicente Grau: GeRTiSS: A Generic Multi-model Surgery Simulator. IS4TH 2003: 59-66 • J. Montagnat and H. Delingette, “Volumetric Medical Images Segmentation Using Shape Constrained Deformable Models,” Proc. First Joint Con5 CVRMed-MRCAS ’97, J. Troccaz, E. Grimson, and R. Mosges, eds. Mar. 1997. • M. Moore and J. Wilhelms, “Collision Detection and Response for Computer Animation,” Computer Graphics (SIGGRAPH ’88), vol. 22, pp. 289-298, Aug. 1988. •   Laks Raghupathi, Laurent Grisoni, Fran?ois Faure, Damien Marchal, Marie-Paule Cani, Christophe Chaillou, "An Intestinal Surgery Simulator: Real-Time Collision Processing and Visualization," IEEE Transactions on Visualization and Computer Graphics, vol. 10, no. 6, pp. 708-718, November/December, 2004.