Presented at the 4 th Workshop on Distributed Supercomputers, New Orleans, March 9-10, 2000

1. An application-based measureof cluster success Mark L. SawleyVisiting Scientist: EPF-Lausanne, SwitzerlandPermanent address: CSIRO Mathematical & Information Sciences, Australia Presented at the 4th Workshop on Distributed Supercomputers, New Orleans, March 9-10, 2000

2. Measure of cluster success Question: What is the best measure of success of a general-purpose central distributed supercomputer (cluster) system? (One) Answer: A wide application community, not just a small number of heavy users This measure is very stringent and is comprised of more conventional aspects, such as performance and availability.

3. Who are the potential users? • An abbreviated history of central parallel machines at the EPFL: • 1986 Cray 1 (2 proc.) • 1988 Cray 2 (4 proc.) • 1994 Cray Y-MP (4 proc.) + Cray T3D (256 proc.) • 1996 SGI Origin2000 (80 proc.) • 1998  Swiss Tx (8  70 proc.) • Potential EPFL cluster community origins: • vector (Crays, NEC) • distributed memory massively parallel (Cray T3D) • NUMA (SGI Origin2000) • workstation

4. Classification of users Require “early adapters” to encourage use by “conservative users” otherwise number of users will be limited.

5. “Technology enthusiasts” • Characteristics: • not interested in application results • main interest in future technology • Needs: • a system (containing hardware + software) • performance irrelevant • no additional software required • access to system gurus • sufficient time (between machine crashes)

6. “Early adapters” • Characteristics: • interested in application results • willing to invest in future technology • Needs: • robust system (hardware + software) • good performance • minimum amount of software (MPI, BLAS, ….) • basic level of user support • dedicated time (for benchmarking)

7. “Conservative users” • Characteristics: • only interested in application results • not willing to invest in future technology • Needs: • user-friendly system (looks like presently-used system) • excellent performance (better than presently-used system) • extensive software (debugging, visualisation, commercial codes, …) • high level of user support • sufficient machine access (to complete large-scale applications)

8. Examples of “early adapters” of the Swiss-T1 • Computational Fluid Dynamics– nsmb (multi-block, finite volume)– fluent (unstructured, finite volume) • –speculoos (object-oriented, spectral element) • Granular flow • – gfc (discrete element method) • Molecular Dynamics • –lautrec (Carr-Parrinello method) • Structural analysis • – crash (object-oriented, finite element) • Plasma transport • – orb (particle-in-cell, finite element)

9. Examples of “conservative users” • Two classes: • Users porting their own codes– need a lot of user support • Users of existing parallel codes • – in-house • – commercial • At the EPFL, these users can potentially come from any of the • 11 different departments (ranging from Mathematics to Architecture). • Both classes can be encouraged by benchmark results.

10. Example of a commercial code benchmarks Fluent CFD code: Suite of 9 different test cases of different size and physical complexity (eg turbulence, combustion) Small class < 100,000 cells Medium class 100,000 – 500,000 cells Large class > 500,000 cells Full details: www.fluent.com/software/fl5bench

11. Computational Fluid Dynamics (nsmb) Code: nsmb is a 2D/3D multi-block compressible flow solver Personnel: Jan Vos (EPFL)

12. Computational Fluid Dynamics (nsmb) AS28G full aircraft configuration, using mesh: 62 unequal blocks 3.5 million cells (~ 2 GB memory required)

13. Computational Fluid Dynamics (fluent) Code: fluent is a commercial 2D/3D unstructured flow solver Personnel: Mark Sawley (EPFL/CSIRO) Kenics static mixer

14. Computational Fluid Dynamics (fluent)

15. Granular flow (gfc) Code: gfc is a 2D/3D Discrete Element Method code Personnel: Mark Sawley & Paul Cleary (CSIRO) Hicom nutating mill Flow from a two-port hopper

16. Granular flow (gfc)

17. Molecular Dynamics (lautrec) Code: lautrec is a first-principles MD code using Carr-Parrinello method Personnel: Alessandro De Vita, Massimiliano Stengel (EPFL) Andrew Canning, Roland Richter (SGI/Cray Research) DNA molecule diamond-graphite transition bovine pancreatic trypsin inhibitor

18. Molecular Dynamics (lautrec) Performance results on Swiss-T1 baby machine using MPI on T-Net. big system 32 water molecues cell dimension: 18.64 a.u. cut-off energy: 90 Ry FFT grid: 112x112x112 medium system 32 water molecules cell dimension: 18.64 a.u. cut-off energy: 30 Ry FFT grid: 64x64x64 small system 108 aluminium atoms cell dimension: 22.92 a.u. cut-off energy: 10 Ry FFT grid: 48x48x48

19. Conclusion of preliminary benchmarks • The initial timing results of application codes on the Swiss-T1 show: • good overall performance due to Alpha ev6 processor • good scalability up to the 12 processors available on the prototype Swiss-T1 baby • substantially better scalability for communication-intensive applications using T-Net compared to Fast- or Giga-Ethernet