1 / 64

Neural Network Verification Part 5: Complete Methods

Neural Network Verification Part 5: Complete Methods. Neural Network Verification. Post. Neural network f. Scalar output z = f( x ). E.g. in binary classification, z = s(y*; x ) – s( y; x ) for y ≠ y*. Property: f( x ) > 0 for all x ∈ X. Complete methods try to disprove the property.

bmelancon
Download Presentation

Neural Network Verification Part 5: Complete Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Neural Network VerificationPart 5: Complete Methods

  2. Neural Network Verification Post Neural network f Scalar output z = f(x) E.g. in binary classification, z = s(y*;x) – s(y;x) for y ≠ y* Property: f(x) > 0 for all x∈ X Complete methods try to disprove the property

  3. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V V1 V2 BRANCH: Split the feasible set 2 or more usually disjoint subsets

  4. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 3 V1 V2 BOUND: Compute upper bounds for each branch h(v) for any feasible v (unsound methods)

  5. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 BOUND: Compute lower bounds for each branch Convex relaxations (incomplete methods)

  6. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 PRUNE: Any lower bounds greater than 0? NO

  7. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 SELECT: Choose a subproblem Say, we choose V2

  8. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 V21 V22 BRANCH: Split the feasible set

  9. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 V21 V22 BOUND: Compute upper bounds Upper bounds of children are smaller than the parent

  10. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 1 -1 V21 V22 BOUND: Compute lower bounds Lower bounds of children are greater than the parent

  11. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 1 -1 V21 V22 PRUNE: Any lower bounds greater than 0? YES

  12. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 1 -1 V21 V22 PRUNE: Any lower bounds greater than 0? YES

  13. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 1 -1 V21 V22 SELECT: Choose a subproblem Say, we choose V1

  14. Branch-and-Bound Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 1 -1 V11 V12 V21 V22 BRANCH: Split the feasible set

  15. Termination – Case I Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 1 -1 V11 V12 V21 V22 We find a counter-example An upper bound that is less than 0

  16. Termination – Case II Post Find v ∈ V such that h(v) ≤ 0 V 2 -3 3 -2 V1 V2 2 1 1 -1 V11 V12 V21 V22 We prove there does not exist v ∈ V s.t. h(v) ≤ 0 All leaf nodes have lower bound > 0

  17. Unified Framework Post All proposed complete methods are branch-and-bound Choice of branching Choice of bounding Choice of selecting (critical for false properties)

  18. Outline • Reluplex • Planet • Input Domain Branch-and-Bound • Result Katz, Barrett, Dill, Julian and Kochendefer, 2017

  19. Example Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 a ain = x1 + x2 1 -1 x1 1 [-2, 2] bin = x1 - x2 z aout = max{ain,0} x2 [-2, 2] 1 -1 -1 bout = max{bin,0} b Prove that z > -5 z = - aout - bout z ≤ -5

  20. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout Infeasible z ≤ -5

  21. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout Relax all non-linearities z ≤ -5

  22. Relaxation Post ain ∈ [l,u] aout = max{ain,0} aout ain l u Replace with convex superset

  23. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ 4 bout ≥ bin, bout ≥ 0, bout ≤ 4 z = - aout - bout Relax all non-linearities z ≤ -5

  24. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ 4 bout ≥ bin, bout ≥ 0, bout ≤ 4 z = - aout - bout z ≤ -5

  25. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout Infeasible z ≤ -5

  26. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} Fix a non-linearity (say b) z = - aout - bout Relax other non-linearities z ≤ -5

  27. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ 4 bout = bin Fix a non-linearity (say b) z = - aout - bout Relax other non-linearities z ≤ -5

  28. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ 4 bout = bin z = - aout - bout z ≤ -5

  29. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 Repeat ain = x1 + x2 Try to fix some non-linearities bin = x1 - x2 aout = max{ain,0} Relax other non-linearities bout = max{bin,0} z = - aout - bout After some attempts, branch z ≤ -5

  30. Branching Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 Choose a non-linearity that is fixed many times bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout z ≤ -5

  31. Branching Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 Choose a non-linearity that is fixed many times bin = x1 - x2 aout = max{ain,0} Split into two bout = max{bin,0} z = - aout - bout z ≤ -5

  32. Branching Post -2 ≤ x1≤ 2 -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 ain = x1 + x2 bin = x1 - x2 bin = x1 - x2 aout = ain aout = 0 bout = max{bin,0} bout = max{bin,0} z = - aout - bout z = - aout - bout z ≤ -5 z ≤ -5

  33. Outline • Reluplex • Planet • Input Domain Branch-and-Bound • Result Ehlers, 2017

  34. Example Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 a ain = x1 + x2 1 -1 x1 1 [-2, 2] bin = x1 - x2 z aout = max{ain,0} x2 [-2, 2] 1 -1 -1 bout = max{bin,0} b Prove that z > -5 z = - aout - bout z ≤ -5

  35. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout Relax all non-linearities z ≤ -5

  36. Relaxation Post ain ∈ [l,u] aout = max{ain,0} aout ain l u Replace with convex superset

  37. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout z ≤ -5

  38. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout z ≤ -5

  39. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Feasible? NO z ≤ -5

  40. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Prune away z ≤ -5

  41. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Feasible? YES z ≤ -5 x1 = 2, x2 = 2, aout = 4, bout = 2

  42. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Feasible? NO z ≤ -5

  43. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Prune away z ≤ -5

  44. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Feasible? NO z ≤ -5

  45. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Prune away z ≤ -5

  46. Branching Post -2 ≤ x1≤ 2 aout = 0 aout = ain -2 ≤ x2 ≤ 2 bout = 0 bout = 0 bout = bin bout = bin ain = x1 + x2 bin = x1 - x2 aout ≥ ain, aout ≥ 0, aout ≤ ain/2+2 bout ≥ bin, bout ≥ 0, bout ≤ bin/2+2 z = - aout - bout Property must be true z ≤ -5

  47. Outline • Reluplex • Planet • Input Domain Branch-and-Bound • Result Bunel, Turkaslan, Torr, Kohli and Kumar, 2018

  48. Example Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 a ain = x1 + x2 1 -1 x1 1 [-2, 2] bin = x1 - x2 z aout = max{ain,0} x2 [-2, 2] 1 -1 -1 bout = max{bin,0} b Prove that z > -5 z = - aout - bout

  49. Bounding Post -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout Relax all non-linearities

  50. Relaxation Post ain ∈ [l,u] aout = max{ain,0} aout ain l u Replace with convex superset

More Related