1 / 20

iDIBS : Reliable and Efficient Distributed Backup

Tam Chantem, Philip Little and Faruck Morcos. iDIBS : Reliable and Efficient Distributed Backup. Improving Peer-to-Peer Backup Systems. Objective : Improve peer-to-peer backup techniques. Approach : Built on DIBS , an existing peer-to-peer backup system. Results : Improved 3 aspects of DIBS:

aimon
Download Presentation

iDIBS : Reliable and Efficient Distributed Backup

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tam Chantem, Philip Little and Faruck Morcos iDIBS: Reliable and Efficient Distributed Backup

  2. Improving Peer-to-Peer Backup Systems • Objective: Improve peer-to-peer backup techniques. • Approach: Built on DIBS, an existing peer-to-peer backup system. • Results: Improved 3 aspects of DIBS: • Reliability, Network Load, and Computation Time.

  3. Erasure Codes iDIBS Peer List iDIBS Peer File Encoding Peer Peer Database

  4. Erasure Codes iDIBS Peer List iDIBS Peer File Encoding Peer Peer Database

  5. a b c d e f g h RS Encoder a d g j i j b e h k k l c f i l Backup and Recovery Encoding and Distributing: Peer File Peer Peer

  6. a b c d e f g h RS Decoder a d j i j b k k l c i l Backup and Recovery Recovering and Decoding: Peer File Dead Peer Peer

  7. ? Data Peer List ? a b c ? ? d e f Peer-List Backups Unmodified DIBS after a crash: Do you have my data? Client

  8. ? Data Peer List ? a Peer List b c ? ? d Peer List e f Peer-List Backups iDIBS after a crash: Do you have my peer list? Client

  9. Data Peer List a b c d e f Peer-List Backups iDIBS after a crash: Client

  10. Thresholds • K – Required number of pieces to recover • N – Total number of pieces transmitted • Redundancy level: • DIBS: N = 2K • iDIBS: Recovery when up to 1 peer is dead

  11. Example Assume: Peers = 4 K = 5 N = 6 Peer 1 Peer 2 Peer 3 Peer 4 Can’t recover!!! So, N = 8

  12. Reduction in Network Utilization

  13. Luby Transform Codes • Reed-Solomon (RS) codes not scalable • Luby Transform (LT) codes • Digital Fountain concept • Probabilistic nature • 15% redundancy needed for successful decoding • Flexible symbol size T, to speed up decoding

  14. Implementation: LT Codes • Encoder and Decoder modules • Encoder: • Calculates redundancy needed • Encodes and splits file depending on T, number of users, and probabilistic parameters • Decoder: • Decodes pieces • Reconstructs file

  15. Luby Transform Codes Encoder/Decoder Performance LT codes vs. RS codes Encoder T=256 Decoder T=256 Processing Time (sec) File Size (Kb)

  16. iDIBS vs. DIBS network utilization behavior • DIBS network utilization is flat. • iDIBS has decreased network utilization. (OH when users<10) • Tradeoff in the theoretical minium is better in iDIBS. • Results shown for T=32, for larger numbers iDIBS plots are better!

  17. iDIBS Advantages Disadvantages LT codes are: 1. Faster than RS 2. Flexible performance given T. Decreased overall Network Utilization. Peer Lists: 1. Allow faster recovery. 2. Increase reliability. LT codes are probabilistic. As T > 256 decoding is less stable LT need a minimum of 15% of extra overhead + ESI to decode. Peer lists induce a small amount of overhead The iDIBS tradeoff

  18. Contribution We contributed to the improvement of DIBS in the following ways: • Increase of performancethrough the introduction of a new encoding technology. (LT codes) • Introduction of this encoding scheme to the application of peer-to-peer backup systems. • More Reliability through redesign of the system recovery scheme. (Peer-Lists) • Reduces Network utilization by changing the philosophy of number of transmitted pieces N and LT codes.

  19. Questions??

  20. Implementation: Timeouts & Thresholds If: Peers > K then N = K + 1;else: PiecesPerPeer = 1;while PiecesPerPeer * (Peers - 1) < K do: PiecesPerPeer = PiecesPerPeer + 1; N = N * K;end if If a peer is offline, recovery is still possible

More Related