1 / 24

Designing Middleware for Volunteer Computing

Exploring the benefits, challenges, and potential of volunteer computing through the use of middleware. Discusses the history, scientific computing paradigms, server performance, credit systems, and the goals of volunteer computing. Explores the limits and goals of the BOINC platform.

thomsonr
Download Presentation

Designing Middleware for Volunteer Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. David P. Anderson Space Sciences Laboratory University of California – Berkeley davea@ssl.berkeley.edu Designing Middleware for Volunteer Computing

  2. Why volunteer computing? home PCs your computers academic business • 2006: 1 billion PCs, 55% privately owned • If 100M people participate: • 100 PetaFLOPs, 1 Exabyte (10^18) storage • Consumer products drive technology • GPUs (NVIDIA, Sony Cell)

  3. Volunteer computing history 95 96 97 98 99 00 01 02 03 04 05 06 GIMPS, distributed.net SETI@home, folding@home commercial projects climateprediction.net BOINC Einstein@home Rosetta@home Predictor@home LHC@home BURP PrimeGrid ...

  4. Scientific computing paradigms Bang/buck Control least most Supercomputers Cluster computing Grid computing Volunteer computing most least

  5. BOINC Climate SETI physics biomedical projects volunteers Joe Jens Alice

  6. Participation in >1 project • Better short-term resource utilization • communicate/compute in parallel • match applications to resources • Better long-term resource utilization • project A works while project B thinks work work project computing needs think think time

  7. Server performance How many clients can a project support?

  8. Task server architecture Work creator Feeder Shared mem MySQL clients Scheduler Transitioner Validator Assimilator File deleter DB purger

  9. Server load (CPU)

  10. Server load (disk I/O)

  11. Server limits • Single server (2X Xeon, 100 Mbps disk) • 8.8 million tasks/day • 4.4 PetaFLOPS (if 12 hrs on 1 GFLOPS CPU) • CPU is bottleneck (2.5% disk utilization) • 8.2 Mbps network (if 10K request/reply) • Multiple servers (1 MySQL, 2 for others) • 23.6 million tasks/day • MySQL CPU is bottleneck • 21.9 Mbps network

  12. Credit

  13. Credit display

  14. Credit system goals • Retain participants • fair between users, across projects • understandable • cheat-resistant • Maximize utility to projects • hardware upgrades • assignment of projects to computers

  15. Credit system • Computation credit • benchmark-based • application benchmarks • application operation counting • cheat-resistance: redundancy • Other resources • network, disk storage, RAM • Other behaviors • recruitment • other participation

  16. Benchmarks not whole story

  17. Limits of Volunteer Computing • How much processing/disk/RAM is out there? • Combinations of resources • Data from 330,000 SETI@home participants

  18. Goals of BOINC • > 100 projects, some churn • Handle big data better • BitTorrent integration • Use GPUs and other resources • DAGs • Participation • 10-100 million • multiple projects per participant

More Related