1 / 43

Accountable systems or how to catch a liar?

Accountable systems or how to catch a liar?. Jinyang Li (with slides from authors of SUNDR and PeerReview). What we have learnt so far. Use BFT replication to handle (a few) malicious servers Strengths : Transparent (service goes on as if no faults) Minimal client modifications

buzz
Download Presentation

Accountable systems or how to catch a liar?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Accountable systems orhow to catch a liar? Jinyang Li (with slides from authors of SUNDR and PeerReview)

  2. What we have learnt so far • Use BFT replication to handle (a few) malicious servers • Strengths: • Transparent (service goes on as if no faults) • Minimal client modifications • Weaknesses: • Breaks down completely if >1/3 servers are bad

  3. Hold (bad) servers accountable • What is possible when >1/3 servers fail? • E.g. 1 out of 1 servers fail? • Let us consider a bad file server • Can it delete a client’s data? • Can it leak the content of a client’s data? • Can it change a client’s data to arbitrary values? • Can it show client A garbage and claim it’s from B? • Can it show client A old data of client B? • Can it show A (op1, op2) and B (op2, op1)?

  4. Fool me once, shame on you; fool me twice, shame on … Hold (bad) servers accountable • Lessons: • Cannot prevent a bad server from misbehaving at least once • We can do many checks to detect past misbehaviors! • Useful?

  5. Case study I: SUNDR • What’s SUNDR? • A (single server) network file system • Handle potential Byzantine server behavior • Can run on an un-trusted server • Useful properties: • Tamper-evident • Unauthorized operations will be immediately detected • Can detect past misbehavior • If server drops operations, can be caught eventually

  6. Ideal File system semantics • Represent FS calls as fetch/modify operations • Fetch: client downloads new data • Modify: client makes new change visible to others • Ideal (sequential) consistency: • A fetch reflects the sequence of modifications that happen before it • Impossible when the server is malicious • A is only aware of B’s latest modification via the server • Goal: • get as close to ideal consistency as possible

  7. A Modify f1 sig1 B Fetch f4 sig2 A Modify f2 sig3 userA sig1 A Modify f1 sig1 B Fetch f4 sig2 sig2 sig3 userB A Modify f1 sig1 B Fetch f4 sig2 A Modify f1 sig1 B Fetch f4 sig2 A Modify f2 sig3 Strawman File System A: echo “A was here” >> /share/aaa File server A Modify f2 sig3 B: cat /share/aaa B Fetch f2 sig4

  8. A Modify f1 sig1 A Modify f1 sig1 B Fetch f4 sig2 B Fetch f4 sig2 A Modify f2 sig3 A Modify f2 sig3 B Fetch f2 sig4 Log specifies the total order A’s latest log: LogA The total order: LogA≤ LogB iff LogA is prefix of LogB B’s latest log: LogB

  9. A: echo “A was here” >> /share/aaa A Modify f1 sig1 B Fetch f4 sig2 A Modify f2 sig3 A A Modify f1 sig1 B Fetch f4 sig2 A Modify f1 sig1 B Fetch f4 sig2 B A Modify f1 sig1 B Fetch f4 sig2 Detecting attacks by the server File server A Modify f2 sig3 B: cat /share/aaa (stale result!) B Fetch f2 sig3

  10. A Modify f1 sig1 B Fetch f4 sig2 A Modify f2 sig3a Detecting attacks by the server A’s latest log: LogA sig1 sig2 A’s log and B’s log can no longer be ordered: LogA ≤LogB, LogB ≤LogA sig3a B’s latest log: A Modify f1 sig1 B Fetch f4 sig2 B Fetch f2 sig3b LogB

  11. What Strawman has achieved • High overhead, no concurrency • Tamper-evident • A bad server can’t make up ops users didn’t do • Achieves fork consistency • A bad server can conceal users’ ops from each other, but such misbehavior could be detected later

  12. … Fork Consistency: A tale of two worlds File Server A’s view B’s view

  13. Fork consistency is useful • Best possible alternative to sequential consistency • If server lies only once, it has to lie forever • Enable detections of misbehavior: • users periodically gossip to check violations • or deploy a trusted online “timestamp” box

  14. SUNDR’s tricks to make strawman practical • Store FS as a hash tree • No need to reconstruct FS image from log • Use version vector to totally order ops • No need to fetch entire log to check for misbehavior

  15. Trick #1: Hash tree Key property: h0 verifies the entire tree of data h0 D0 h1 h2 h3 D2 D1 D3 h4 h5 h6 h7 h8 h9 h12 h10 h11 D7 D8 D10 D11 D4 D9 D12 D5 D6

  16. Trick #2: version vector • Each client keeps track of its version # • Server stores the (freshest) version vector • Version vector orders all operations • E.g. (0, 1, 2) ≤ (1, 1, 2) • A client remembers the latest version vector given to it by the server • If new vector does not succeed old one, detect order violation! • E.g. (0,1,2) ≤ (0,2,1)

  17. SUNDR architecture SUNDR server-side • block server stores blocks retrievable by content-hash • consistency server orders all events SUNDR client userA Untrusted Network block server userB consistency server SUNDR client

  18. SUNDR data structures: hash tree • Each file is writable by one user or group • Each user/group manages its own pool of inodes • A pool of inodes is represented as a hash tree

  19. Hash files data1 • Blocks are stored and indexed by hash on the block server i-node data2 Metadata 20-byte File Handle H(data1) data3 iblk1 H(data2) H(data3) H(iblk1) data4 H(data4)

  20. 2 3 4 20-byte 20-byte 20-byte Hash a pool of i-nodes • Hash all files writable by each user/group • From this digest, client can retrieve and verify any block of any file i-node 2 i-table i-num 20-byte digest i-node 3 i-node 4

  21. Lookup “/” Lookup “/share” digest digest digest 4 3 2 2 4 3 3 2 4 20-byte 20-byte 20-byte 20-byte 20-byte 20-byte Fetch “/share/aaa” /: Dir entry: (share, Superuser, 3) GroupG: /share: Dir entry: (aaa, UserA, 4) SUNDR FS Superuser: SUNDR State UserA: digest How to fetch “/share/aaa”? … UserB:

  22. SUNDR data structures:version vector • Server orders users’ fetch/modify ops • Client signs version vector along with digest • Version vectors will expose ordering failures

  23. Version structure A - 1 B - 1 G - 1 A A - 1 B - 2 G - 2 B • Each user has its own version structure (VST) • Consistency server keeps latest VSTs of all users • Clients fetch all other users’ VSTs from server before each operation and cache them • We order VSTA≤VSTB iff all the version numbers in VSTA are less than or equal in VSTB ≤ VSTA VSTB Digest A Digest B Signature A Signature B

  24. B A-0 B-1 DigB A A-1 B-1 DigA A A-1 B-1 DigA A A-1 B-1 DigA B A-1 B-2 VSTA≤VSTB DigB B A-1 B-2 DigB Update VST: An example Consistency Server A A-0 B-0 A DigA A: echo “A was here” >> /share/aaa B B A-0 B-1 DigB B: cat /share/aaa

  25. A A-0 B-0 ≤ DigA B A-0 B-2 A’s latest VST and B’s can no longer be ordered: VSTA≤ VSTB, VSTB≤ VSTA DigB B A-0 B-2 DigB Detect attacks A Consistency Server A A-0 B-0 DigA B A-0 B-1 DigA A A-1 B-1 A: echo “A was here” >> /share/aaa B DigA B A-0 B-1 A A-1 B-1 DigB DigA B: cat /share/aaa (stale!)

  26. Support concurrent operations • Clients may issue operations concurrently • How does second client know what vector to sign? • If operations don’t conflict, can just include first user’s forthcoming version number in VST • But how do you know if concurrent operations conflict? • Solution: Pre-declare operations in signed updates • Server returns latest VSTs and all pending updates, thereby ordering them before current operation • User computes new VST including pending updates • User signs andcommits new VST

  27. update: A-1 update: B-2 B A-0 B-1 A-1 DigB A A-1 B-1 DigA A A-0 B-0 A-1 A A-1 B-1 A-1 B-2 DigA DigA A-1 B A-1 B-2 VSTA≤VSTB DigB A-1 B-2 B A-1 B-2 DigB A-1 B-2 Concurrent update of VSTs Consistency Server A B A A-0 B-0 DigA B A-0 B-1 DigB A: echo “A was here” >>/share/aaa B: cat /share/bbb

  28. SUNDR is practical Seconds

  29. Case study II: PeerReview [SOSP07]

  30. Motivations for PeerReview • Large distributed systems consist of many nodes • Some nodes become Byzantine • Software compromise • Malicious/careless administrator • Goal: detect past misbehavior of nodes • Apply to more general apps than FS

  31. Challenges of general fault detection • How todetect faults? • How toconvince others that a node is (not) faulty?

  32. Overall approach • Fault := Node deviates from expected behavior • Obtain a signature for every action from each node • If a node misbehaves, the signature works a proof-of-misbehavior against the node

  33. 0 X Can we detect all faults? 100101011000101101011100100100 • No • e.g. Faults affecting a node's internal state • Detect observable faults • E.g. bad nodes send a message that correct nodes would not send A C

  34. Can we always get a proof? I sent X! A • No • A said it sent X B said A didn‘t C: did A send X? • Generate verifiable evidence: • a proof of misbehavior (A send wrong X) • a challenge (C asks A to send X again) • Nodes not answering challenges are suspects X ? B I neverreceived X! ?! C

  35. PeerReview Overview • Treat each node as a deterministic state machine • Nodes sign every output message • A witness checks that another node outputs correct messages • using a reference implementation and signed inputs

  36. PeerReview architecture A's witnesses • All nodes keep a log of their inputs & outputs • Including all messages • Each node has a set of witnesses, who audit its log periodically • If the witnesses detect misbehavior, they • generate evidence • make the evidence avai-lable to other nodes • Other nodes check evi-dence, report fault C D E M M A M B A's log B's log

  37. B's log H4 Recv(M) H3 Hash chain Send(Z) H2 Recv(Y) H1 H0 Send(X) PeerReview detects tampering Message Hash(log) B A • What if a node modifies its log entries? • Log entries form a hash chain • Signed hash is included with every message Node commits to having received all prior messages ACK Hash(log)

  38. "View #1" "View #2" H4' H4 OK H3 Not found H3' Create X Read X H2 OK H1 H0 Read Z PeerReview detects inconsistency • What if a node • keeps multiple logs? • forks its log? • Witness checks if signed hashes form a single chain

  39. PeerReview detects faults • Witness audits a node: • Replay inputs on a reference implementation of the state machine • Check outputs against the log Module A State machine Module B Network Log Module A Module B Input if ≠ =? Output

  40. PeerReview‘s guarantees • Faults will be detected (eventually) • If a node if faulty: • Its witness has a proof of misbehavior • Its witness generates a challenge that it cannot answer • If a node is correct • No proof of misbehavor • It can answer any challenge

  41. PeerReview applications • Metadata corruption • Incorrect access control • App #1: NFS server • Tampering with files • Lost updates • App #2: Overlay multicast • Freeloading • Tampering with content • App #3: P2P email • Denial of service • Dropping emails

  42. PeerReview’s performance penalty 100 80 • Cost increases w/ # of witnesses per node W 60 Avg traffic (Kbps/node) Baseline traffic 40 20 0 Baseline 2 1 5 3 4 Number of witnesses

  43. What have we learnt? • Put constraints on what faulty servers can do • Clients sign data, bad SUNDR server cannot fake data • Clients sign version vector, bad server cannot hide past inconsistency • Fault detection • Need proof of misbehavior (by signing actions) • Use challenges to distinguish slow nodes apart from bad ones

More Related