Consistent Terminology, Consistent Results

1. Consistent Terminology, Consistent Results Introduction and Definitions

2. Agenda –Introductions and Definitions • Design integrity – A working definition • Why is this important? • A tale of two designs • Has anything changed? • Why should I care? • The miracle cloud • The alternative • Overview • Implications • Additional Definitions • Summary

3. Definition and Goal • Design – The invention and disposition of the forms, parts, or details of something according to a plan. (AH dictionary online) • Integrity – The state of being unimpaired; Soundness; The quality or condition of being whole or undivided; completeness (AH) • This seminar is intended to talk about techniques and issues that ensure the soundness and completeness of both the end product and the means used to produce it.

4. Design 1 – Radarsat 1 ACP

5. Radarsat 1 ACP Overview • Program dates: late 1990 – late 1992 • Specifications • Processor: 8 MHz 80386/80387 • Memory:128 k x 8 SRAM, 128kx8 EEPROM, 16k x 8 PROM • Interfaces: A/D (16), D/A (4-12), Synchronous serial (3 input, 3 output), RS-232 GSE • Function: Attitude control processor for the RADARSAT1 satellite • Logic Implementation MSI logic + PALs (16L8, 16R6, 16R8) • Additional functions (cross-strap, control) external

6. PAL Reminder

7. Design 2 - Command Telemetry Board (CTB)

8. CTB Overview • Program Dates: early 2001 – late 2003 • Specifications: • Processor: RTX2010 (16 MHz) • Memory: 4M x 8 (random), various FIFOs (16k x8) as necessary • Interfaces • MIL-STD-1553B • Synchronous Serial (command / telemetry) • Analog input • High-level discrete (output) • Low-level discrete (input / output) • Functionality: S/C command/telemetry (level 0 and active) • Logic Implementation: 4 54SX32S FPGAs • Additional resources (in same box): RAD 750, Mass Memory, Instrument Interface Card

9. What’s Changed? • Capability / Complexity • Logic Density • Specificity • RADARSAT (1 small specification with interface focus) • CTB (1 large specification with interface, s/w, operations focus) • Software Centricity • Initial Errors • RADARSAT: 3 jumpers; 1 PAL design change • CTB: 14 FPGA revisions • 2 spec change • 5-6 mistakes • 6-7 data dependency

10. What’s Not Changed? • Overall program schedules • Proportional budget • Expectation of correctness • Pain from mistakes

11. What Explains the Difference? • Engineers aren’t as capable? – Insulting! • Everything is just more complex? – Copout! • Methodology? • Methodology hasn’t changed • Always inadequate, we just got lucky • Adequate for old designs, no longer adequate • Methodology has changed • Used to be adequate, but we lost the recipe • Design philosophy of systems has changed? • Predicated on maximum flexibility • Expectation of extreme complexity • Over-specification – almost impossible to meet

12. What Do These Examples Illustrate? • The incidence of initial correctness for designs seems to be decreasing • Design changes seem to be more common • Problems late in the verification/validation cycle seem to be more frequent • Perhaps a combination of the factors presented explains this, but … • Desired complexity is not going to decrease • Budgets are not going to get bigger • The expectation of excellence isn’t going to go away • The only solution is to develop and improve a consistent methodology for implementing robustly designed products • Based on basic principles • Applicable to a variety of conditions

13. Why Should I Care? • Why do I work? • Self-actualization (fun, monetary reward, interest) • Why do people want us to work for them? • They need what we produce • What do people want engineers (especially in Aerospace) to produce? • A quality product that satisfies the customer’s needs • How do they want us to produce such a product? • Consistently and efficiently

14. The Layman’s View – The Miracle Cloud

15. The Miracle Cloud Method • Note that too many engineering schools teach this approach without meaning to • Advantages to the miracle cloud method • Total creative freedom • Disadvantages to the miracle cloud method • Product quality is variable • Team makeup dependent • Team mood/morale dependent (Monday morning car) • Luck dependent • Product is not produced in a repeatable manner • Product is not produced in an efficient manner • Result • Quality low • Customer Satisfaction Low

16. How Do We Replace the Miracle Cloud? • Provide structure to the development effort • Evaluate the effort and the product produced • Improve the effort and the product • Repeat

17. Definitions of Importance • From Q9000-2000 • Process – A set of interrelated and interacting activities which transforms inputs to outputs [in our case ideas to devices] • Product – The result of a process

18. Implications From These Definitions • If we want a consistent product, we must have a consistent process • If we want to improve a product, we must improve the process • If our company has no (or inadequate) process and we must produce a quality product, then we must establish a process [personal responsibility] • Developing, imposing, and improving a process is not an end (in and of itself) it is only a means to an end

19. A Model for Discussing the Design Process

20. Notes on the Model • Feedback / iteration are not shown for clarity • Model may be recursive • Board development process includes FPGA requirement definition, FPGA development, instantiation, etc. • Board development process includes the FPGA validation product • Successes and failures are cumulative • Good requirements + successful development => successful instantiation • Bad requirements + failed development => failed instantiation • Complexity multiplies • Complex requirements increase design complexity which, in turn, increases verification complexity • Processes are absolute gates to the next stage of development

21. Implications From the Model • All processes must be addressed at all levels of design [there are no shortcuts!] • Does not imply same formality at all levels • Does imply same rigor at all levels • Up front work on requirements is essential! • Must provide adequate time and money • Must gain team buy-in to the process* • Benefits compound throughout the rest of the activities • Simplicity is an essential virtue • Complexity inevitably multiplies • A product is not qualified until both verification and validation are complete

22. Additional Useful Definitions (courtesy of Q9000-2000) • Specification – A document* stating requirements, needs, or expectations that are obligatory • Quality – The degree to which a set of inherent characteristics fulfill requirements • Customer satisfaction – Customer’s perception of the degree to which the customer’s requirements have been fulfilled • Verification – Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled • Validation – Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled • Objective evidence – Data supporting the existence or verity of something • Continual Improvement – recurring activity to increase the ability to fulfill requirements • Note the importance of Requirements

23. Summary • I have no assurance that my product will have consistent quality without: • Well-defined requirements • A well planned approach to implementing the requirements • A clearly defined plan for verification and validation of the requirements • The ability to improve the process that produces the product • Without quality product, customer satisfaction is impossible • Without customer satisfaction, I won’t work! • Therefore, I must care about ensuring design integrity