1 / 42

Parallel Computing Models & Techniques

Parallel Computing Models & Techniques. About Me. Microsoft MVP Intel Blogger TechEd Israel, TechEd Europe Expert C++ Book http ://AsyncOp.com http://Asaf.Shelly.co.il. Parallel Computing. Multi-Core Distributed Systems SOA & WebServices Transaction, Session, Queue, Event, Interrupt

Download Presentation

Parallel Computing Models & Techniques

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Parallel ComputingModels & Techniques

  2. About Me • Microsoft MVP • Intel Blogger • TechEd Israel, TechEd Europe • Expert C++ Book • http://AsyncOp.com • http://Asaf.Shelly.co.il

  3. Parallel Computing • Multi-Core • Distributed Systems • SOA & WebServices • Transaction, Session, Queue, Event, Interrupt • User Experience over User Interface • Maximize performance: No Free Work Unit • Best performance: No I/O Wait

  4. Advantages of Multi-Core • Low Power Consumption • Extended battery life • Less heating • Smaller and lighter devices • Software replaces custom hardware!

  5. Signaling In Hardware Request: Read Address Error Detection RAM Wait: Preparing Data Response: Data CPU

  6. Signaling In Hardware Interrupt : Data Pending Error Detection RAM Interrupt : Processing Complete CPU

  7. Software Locks? • Locks Are BAD! • By design a lock is forcing serial work • Using a resource on a single core • Use a lock only when you want to use 1 core at a time and eliminate parallel work • Locks can be used on single steps for example entrance to a queue

  8. Locks Are BAD! • Can you find the bug?? Lock( MUTEX_A ) Buffer_A [ 12 ] = 23; // here Buffer_A [ 12 ] is 57 !!!! Unlock( MUTEX_A ) • Would you find it with a code review?

  9. Lock = Stop

  10. Need Lock-Free Solutions!

  11. Protecting A Resource • Lock as way to share ownership • Using a single owner • Owner Thread • Owner Task • TPL Agent • Device Driver • Owner Service

  12. Asynchronous Work Without Locks • Phone as Synchronous System • Phone as Asynchronous System • Mail and Email System • Order Pizza

  13. Unprotected Parallel Access To Data • Two Writers or Writer and Reader Writer Writer Reader A A A A A A A A A A A A A A A A A

  14. Race Condition - Location • Two Writers or Writer and Reader Writer Writer Reader A A A A A A A A A A A A A A A A A

  15. Race Condition – Timeframe • Collision over the same communication line Writer Writer Reader A

  16. Race Condition - Sequence • Bugs in Parallel Pipeline 123 123 Add X Add ‘1’ To ASCII Clear Buffer Add ABCDE Add X Add ABCDE Clear Buffer Add ‘1’ To ASCII 123ABCDE ABCDE ABCDEX BCDEFY X TCP, CJP, PF

  17. Race Condition Solutions • Wave-In Signal – Manager (ex. USB BUS) • Pass Ownership (Token Ring, MUTEX) • TDM • Burst Write, Retry Read (ex. SeqLock, Reader-Writer Lock, Network Layer 2) • Write and Verify • Queue • Transaction – A Sequence

  18. Serial Problem with Communication • Transaction based Ping-Pong Computer Packet Request USB Device Packet Data Acknowledge Packet Request Packet Data Acknowledge

  19. Parallel Solution for Ping-Pong • Collected Transaction Computer USB Device Request List Packet Data A Packet Data B Packet Data C Retransmit B Packet Data D Ack List

  20. Cancel Operation • Search For File Computer USB Device Request List Packet Data A Packet Data B Packet Data C Acknowledge A Packet Data D Abort Data found in B Packet Data E Acknowledge

  21. Object Oriented Design • Definition Of Objects • Object Relations • Object Reusability • Object Management • Object Oriented Block Diagram • Object Oriented System Design • Avoid “Spaghetti Code”

  22. Procedural Design • Definition Of State • Procedure Relations • Procedure Reusability • Flow Control Management • Poor Block Diagram • Limited System Design • Avoid “Spaghetti Flow”

  23. Business Logic User Interface Infrastructure Parallel Serial Parallel Good Application Design

  24. Asp.Net Browser Web Server Parallel Serial Parallel Good Application Design

  25. Queue • Pass Data Without Using Lock • Full Asynchronous Operation • Event With Data • Event With Priority • Event With Destination • Structured Event vs. Stream

  26. Flow Control • Keep Internal State • Object State • Execution Phase • Collection of State as System State • System State for Debug

  27. Task Management • Stack – Hardware Accelerated Management • Fork • Software Stack Management • Session • Task Groups

  28. Software Dispatcher • .Net Parallel Extensions

  29. Network Dispatcher Load Balance Front End • Load Balancing Firewall Firewall Firewall Load Balance Front End

  30. Hardware Dispatcher • 10 Gbps Network Switch 2.67 GHz CORE 2.67 GHz CORE 2.67 GHz CORE 2.67 GHz CORE 10 GHz Network Dispatcher / MUX

  31. Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker GUI Dispatcher Business Task Logic Interface Application Dispatcher Cloud Dispatcher • Microsoft Server 2008 HPC

  32. Parallel Memory?

  33. Task Oriented Design Operation: Setting up a Tent A Task: locate items in storage B Task: carry items to build site C Task: use items to build tent

  34. A B C A B C Execution Timeline Locate Carry Use Output Wires Fabric A B C Pole Time

  35. A B C A B C Horizontal Division A B C Time

  36. A B C A B C Vertical Division A B C Time

  37. Resource Partitioning

  38. Force Duplication • Entire Process • Sharing Resources • Flow Barriers • Simple to implement • Simple Affinity • Simple Priority • No Optimization

  39. Pipeline • Functional • Resources Ownership • Communication Barriers • Requires Design • Affinity Planning • Priority Planning • Optimization

  40. Super Networks and Grids • Multiple Reads • Multiple Writes • Replication Time • Replication Overhead • Network Consistency • Data Snapshot • Real Time

  41. Super Networks

  42. Thank You

More Related