Report of the Auditability Working Group

1. DRAFT Report of the Auditability Working Group David Flater National Institute of Standards and Technology http://vote.nist.gov

2. Outline Presentation Charge to the working group The goal of software independence (SI) What was actually required in the 2007 TGDC draft Alternatives to SI and their consequences Paper, voter-verification, and accessibility Effectivity concerns 3 options Debate TGDC and EAC discussion Resolutions (choose an option)

3. Charge to the working group Alternatives to Software Independence (SI) – EAC directs the TGDC to develop draft requirements for audit methods to achieve the goal of Software Independence (SI). The goal is to develop requirements for the auditability of the election system without requiring a specific technology. The starting point for these requirements should be the work already completed by NIST on alternatives to SI.

4. The SI rationale (abridged) The following is not the entire SI rationale, but it is the acid test that distinguishes SI from other forms of auditability Accept as plausible that there could be one rogue or coerced software engineer in each independent supplier of voting equipment to the jurisdiction Alternately, that each supplier relies on insecure COTS software that a third party can exploit, or that common mode failures exist, or etc. All electronic records potentially compromised If there are no other records, then it is not possible to compare records to audit the result The goal of SI, as abridged: mitigate this threat (and others that are easier)

5. Mitigation: independent voter-verifiable records Independent records enable a meaningful audit Voter-verification establishes independent validity Validated records must be protected from modification Paper records suffice Direct and indirect verification Ballots dropped into ballot box More difficult to achieve wholesale compromise of paper records without detection Alternatives that mitigated the threat without using paper were not prohibited in the 2007 TGDC draft

6. IVVR versus paper What the 2007 TGDC draft actually required Either independent voter-verifiable records (IVVR), or "Innovation class submission" Intent: the term IVVR was introduced specifically to avoid mandating paper Extent: paperless solutions are still researchy From absence of example, cannot conclude: That the requirements are more restrictive than necessary to achieve the goal That no conforming paperless solution can possibly exist Do not have working group consensus on these assertions

7. Alternatives and consequences Electronic Independent Verification Devices (e.g. VoteGuard) At best incomplete response to the rogue programmers threat Parallel testing Arms race between complexity of testing and complexity of evading detection Cannot be required in the VVSG Punts the problem to poll workers Software assurance Would require invasive, expensive changes to the development process and all-new systems End-to-end crypto still a research topic Unknown unknowns ("innovation class")

8. Tried a different approach Previous state: auditability = SI Suggested new state: auditability = ability to do an automated, independent recount Automated, because manual counting is inaccurate Independent, so that it is a meaningful audit Want something comparable to shipping opscan ballots to neighboring county Falls short of the SI goal if voter-verifiability is not included Paperless approaches = IVD At best incomplete response to the rogue programmers threat How much does "independent" entail Taking verification off the critical path or making it "random"

9. Paper, voter-verification, and accessibility There have been misunderstandings about what exactly the 2007 TGDC draft required and did not require Paper record accessibility requirements were intended to be more general (i.e. stronger) than in VVSG 1.0 "Software independence" maybe conveyed that not allowed to use software for audio readback; that was not the intent 2007 TGDC draft reflected a difficult compromise Identical experience for every voter is infeasible Prohibiting or limiting voter-verification would not be a win Absence of conforming implementations raised objections Rejecting paper entirely versus requiring paper record accessibility If there is agreement to pursue some alternative to SI, then agreement on reasons for rejecting SI is not required If there is not agreement to pursue some alternative to SI, then a better compromise has not yet been identified

10. Effectivity concerns There have been misunderstandings about the impact of VVSG 2.0 on already-deployed systems VVSG 2.0 intended to be "forward-looking" for new certifications after some date EAC determines the date once new guidelines are approved Certificates issued under previous versions of the guidelines will not be revoked automatically when new guidelines are approved No mandate to retrofit or replace already-deployed systems "Worst" case: Assuming that a jurisdiction has [voluntarily adopted] a law that deployed systems must comply with latest EAC guidelines to be used in an election—approval of VVSG 2.0 is not imminent V = Voluntary

11. Option #1 Endorse one or more of the existing paperless alternatives Different alternatives have different implications and consequences Implied policy decisions IVD: reject the rogue programmers threat or accept an incomplete mitigation Parallel testing: accept difficult and/or incomplete procedural mitigation that is outside the scope of EAC certification Software assurance: commit to invasive, expensive changes to the development process and all-new systems Defining a higher-level auditability concept requires relaxing one of the constraints Otherwise auditability = SI

12. Option #2 Conclude that it was all a big misunderstanding Goal of SI + no mandate for paper is what we had in 2007 Paper ballot accessibility requirements—as intended No manual paper ballot handling (Acc-VS) Alternative format verification of complete paper ballot print content Accept that an example of a conforming system need not exist yet No mandate to retrofit or replace Engage Standards Board, Board of Advisors during the process Refocus on communication, first impressions

13. Option #3 No misunderstanding—confirm the previous result Accept the SI argument; accept the SI conclusion Market shifting to opscan Paper ballot accessibility requirements—as intended Manufacturers reportedly are responding to paper handling and readback requirements Fighting the previous battle?