Document Evaluation Process May 2005 Revision

1. Document Evaluation ProcessMay 2005 Revision 8T8L May 2005 Revision

2. Upon successful completion of this topic, you should be able to: Describe the preparation for onsite document review Discuss the ORR/RA principle of sampling versus auditing Discuss the complete trail sampling technique Discuss good Team Member record / note keeping practices related to records review 8T8L May 2005 Revision

3. Review Preparation Flow Chart Requirements Objective Core Orders, Codes, Requirements Criteria Standards, Regulations Identification of Required Program & Processes Identification of important implementing Procedures / Policies Identification of important Systems, Structures, Components Identification of important Job Factors 8T8L May 2005 Revision

4. Review of Procedures • What procedures* are required? • * All documented, implementing mechanisms . . . • Implement Regulatory Drivers . . . • Programs and Processes . . . • Implement Commitments . . . • DSA/SAR / TSR / Accident assumptions; • BIO / JCO / FHA, and EIS; • List A/B, WSS, S/RIDs; • Company / Operations Office procedures & requirements . . . • Control Procedures, those not specifically required, but needed because of complexity, variability, risk, quality: • Control and guide ops and activities to achieve desired outcome 8T8L May 2005 Revision

5. Review of Procedures (continued) • Document Reviews do not have to be line by line copies of Order or regulatory driver, but must: • Detail the mechanisms to fulfill the intent and requirements of the regulatory/contract driver . . . • Detail the mechanisms and requirements to adequately control and document the activity . . . • Focus on implementation . . . 8T8L May 2005 Revision

6. Review of Procedures (continued) • Define the responsible individual(s) for each specified requirement . . . • Be adequately communicated to and understood by those identified individuals . . . • Comply with form and content guidance consistent with department and local expectations . . . 8T8L May 2005 Revision

7. Programs to Control Procedure Adequacy • Review system: • Procedures are adequate? • V&V in the field? • Configuration control: • Procedures relate to systems; commitments identified? • Formal change control? • Adequate supervisory control: • Ensure adequate implementation & performance of procedures? • Issue and use procedures ensure only current revision in use? • Training / Qualification and Assessment programs: • Ensure procedures are adequately understood? 8T8L May 2005 Revision

8. Top Tier Requirement Performance Review Flow DOE / Operations Office Requirement Requirement Specific (what) Contract Requirements Company Policy / Procedure Manual Implementation Specific Project Procedures (how) Line Instructions Result Project Records

9. Sampling • Not a complete document audit . . . • Sample and expand if needed . . . • Sampling may include complete vertical slice: • Engineered change; • USQ screening; • Work package; 8T8L May 2005 Revision

10. Your Notes • Detailed enough to reconstruct . . . • Be able to describe the hierarchy and flow of requirements from highest directive to implementation . . . 8T8L May 2005 Revision

11. Step 1 -- Preparation • At the programmatic or activity level select the target document(s) for review: • Which are the best indicators of the program? • Which are important to: • Environment, safety and health? • Safeguards and security? • Mission? • Prioritize documents to review . . . 8T8L May 2005 Revision

12. Step 1 -- Preparation(continued) • Communicate with site counterpart to determine: • appropriate site documents; • custodians & control mechanisms of documents. • Communicate with Team: • avoid duplication of review; • ensure that team members are aware of documents that may affect their review, and don't assume they know. • Records of Performance comply with Requirements 8T8L May 2005 Revision

13. Step 2 --Document Review • Read and understand the documents . . . • Determine if document contains required information, from developed criteria . . . • Determine how document is controlled . . . 8T8L May 2005 Revision

14. Step 2 -- Document Review (continued) • Determine if document is consistent with facility and activities: • Equipment/hardware/software; • Personnel; • Higher tier commitment; • Related programmatic and performance requirements. • Records of performance comply with Requirements? 8T8L May 2005 Revision

15. Step 3 -- Follow-up • Check beyond the document evaluation: • Training; • Controls to ensure procedures are followed; • Document configuration control. • Check compliance of base level standards to codes, standards, regulations . . . • Consider for interview topics . . . 8T8L May 2005 Revision

16. Step 4 -- Reporting • In addition to the basic process reporting: • Record the information relating to the review criteria; • Check reference information; • Integrate information gained into observation and interview processes; • Discuss conclusions with Team Members, as appropriate; • Meet expected Team Report and Forms 1 and 2 format. 8T8L May 2005 Revision

17. Summary… • Does the documentation accurately and completely implement the requirements: • Contractual? • Authorization Basis? • Programmatic? • Performance? • Is the documentation being implemented and requirements met? • Is “Flow-Down” consistent, complete, and accurate? 8T8L May 2005 Revision