Arguments From Experience: The PADUA Protocol

1. Arguments From Experience: The PADUA Protocol Maya Wardeh, Trevor Bench-Capon and Frans Coenen Department of Computer Science University of Liverpool

2. A Strange Beast • It is warm blooded, has hair, lays eggs, does not suckle its young. Is it a mammal? • Albert: It can’ be a mammal because it lays eggs • Bruce: I’ve seen mammals that lay eggs. And it does have hair • Albert: But mammals with hair also suckle their young • Bruce: All hairy, warm blooded animals are mammals • Albert: So I suppose we must say it is • Albert and Bruce argue on the basis of mammals they have seen • Their experiences have been very different • There may be no right answer – yet!

3. PADUA • PADUA – Protocol for Argumentation Dialogue Using Association Rules • A dialogue game to argue about classification • Arguments are taken directly from a data base of past examples using data mining techniques

4. Arguing from Experience • Most dialogue systems are based on belief bases • Participants use facts and rules to construct their arguments • PADUA uses examples directly • Participants have a data base containing collections of instances representing their past experience • Resembles case based rather than rule based reasoning

5. Experiences May Differ • Participants will have seen different samples. • Geographical: all swans are white in the Northern hemisphere • Exceptions may be only rarely encountered: insufficient support in some DBs • Sample may be abnormal: in law, only hard cases seen at highest level of appeal

6. Advantages • No knowledge representation bottleneck • No advance commitment to a theory • Can deal with gaps and conflicts • Pools experience

7. Dialogue Moves • Participants point to features of the current case which are reasons why it should (or should not) be classified in a certain way • Participants respond by citing other features which provide reasons to challenge the classification

8. Moves Based On Belief Bases • Claim P: P is the head of some rule • Why P: Seeks the body of rule for which P is head • Concede P: agrees that P is true • Retract P: denies that P is true • P since S: A rule with P head and S body

9. Persuasion in Belief Base Systems • A Has a Rule with P as head, but one literal Q in the body is unknown: B shows that Q is true. • B gives A a rule with P as head and body S. A already believes S • A is shown to have an inconsistency: retraction enables P to be shown

10. Moves in Case Based Reasoning • Citing a case: • A past case which shares features with the current case and had the desired outcome • Distinguishing a case: • features in the past case missing from the current case • Features in the current case missing from the past case • Counter Example • A past case which shares features but had a different outcome • Arguments from Experience have many similarities

11. Moves in Argument from Experience • Citing reasons: • Features in the current case which are typically associated with the desired classification, C • Distinguishing • An additional feature which typically identifies an exception • A feature which Cs typically have but which is not present • A feature which increases confidence in the classification • Counter Example • Features in the current case which are typically associated with a different classification, not C

12. PADUA Protocol - Basics PADUA Scenario Instance Case Class C1 Class C2 (C1) A1: P suggests Q A1 A2 A2: P’suggests Q’

13. PADUA Protocol - Basics PADUA Scenario PADUA Moves 1 1: Propose Rule P is a reason for C 2: Distinguish 3: unwanted consequences 6 2 It need not be S P and q is a reason for not C 4: Counter Rule 5: Increase Confidence 6: Withdraw unwanted consequences 3 Cs are not Q 5 it would be more a C if it were R PADUA Protocol P’’ is a reason for not C 4

14. Strategies • The protocol offers a lot of scope for choice: • Which move to make? • Introduce a new association or refine an existing one? • Which association to propose? • Best or just a good one? In terms of confidence or support?

15. Strategies • We have different strategies according to: • Aim: establish a rule or critique opponents (build versus critique) • Persistence: concede when reasonable or only when no argument left (agreeable versus disagreeable) • Different strategies give rise to different flavours of dialogue: • Build + Disagreeable more like persuasion • Critique + Agreeable more like deliberation

16. Experiments • We have experimented with a number of Data Sets: • Poisonous Fungi • US Senators voting records (ESQUARU 2007) • Welfare Benefits (COMMA 2008)

17. Welfare benefits • Large numbers of cases • Lay adjudication: many (often inexperienced) adjudicators • A high (often 20-30%) error rate is typical • Particular clerks and offices may make systematic misinterpretations • PADUA can act as a moderation meeting, allowing debate over classifications drawn from different adjudication sources

18. Conditions for Benefit • Age condition: “Age appropriate to retirement” is interpreted as pensionable age: 60+ for women and 65+ for men. • Income condition:“Available income” is interpreted as net disposable income, rather than gross income, and means that housing costs should exceed one fifth of candidates’ available income to qualify for the benefit. • Capital condition: “Capital is inadequate” is interpreted as below the threshold for another benefit. • Residence condition: “Resident in this country” is interpreted as having a UK address. • Residence exception: “Service to the Nation” is interpreted as a member of the armed forces. • Contribution condition: “Established connection with the UK labour force" is interpreted as having paid contributions in 3 of the last 5 years.

19. Results • Given two databases, each containing a significant proportion of wrongly decided cases based on different systematic errors, correct classifications can be reached • While this is true for errors concerning most attributes, success is markedly less when mistakes relate to the contribution condition

20. Intermediate Predicates • These are legal concepts, which must be satisfied for the law to apply • Need to be defined in terms of observable facts • Some can be unfolded into observable facts • Others need to applied on the balance of consideration of a number of factors • These last present particular problems • E.g. the contribution condition in the example • Also true for other machine learning and data mining systems (e.g. Mozina et al)

21. Nesting Dialogues • If we are aware of intermediate predicates which do not unfold appropriately into sufficient conditions, we can nest a dialogue to decide this issue within the main dialogue • This handles the contribution problem • Confirms other work in AI and Law in which issue based classification is more accurate than holistic approaches

22. Summary • PADUA offers a novel kind of persuasion dialogue, based on examples rather than a belief base. The result has more in common with case based than rule based reasoning • It avoids the need for knowledge representation effort • The databases are not shared, enabling distinctive features of particular DBs to be identified and maintaining some level of privacy • Where issues can be identified and resolved in preliminary dialogues, accuracy can be improved

23. The Talk Is Finished