Why Use TRIMS? The Importance of the Technical Risk Identification and Mitigation System

1. Why Use TRIMS? The Importance of the Technical Risk Identification and Mitigation System Presented By: Ralph Sickinger BMP Center of Excellence July 2005 Anne Marie T. SuPrise, Ph.D. Director BMP Center of Excellence annemarie@bmpcoe.org 301.405.9990

2. Do You Know This Person? It is halfway through year 1 of the All-Purpose Cruise Missile (APCM) program, and “Pete the Program Manager” finds himself facing a lot of decisions. Time is money, and he wants this development program to be completed as soon as possible (under three years), so that the cruise missile can be fielded and help keep the world safe from tyranny. What can he do to shorten the schedule? Over the next 6 months, Pete will make the following decisions: • Eliminate the Design Reference Mission Profile (DRMP) – The mission goals for this cruise missile are pretty straightforward and well understood, so the time spent developing and documenting the DRMP would be a complete waste. • The design engineers want the body of the cruise missile to be fabricated from a new high-tech plastic-composite which can be easily formed; this should result in less weight and cheaper production costs. Pete approves the design. • In order to keep personnel costs down, Pete specifies a minimum number of personnel in the design contract: mostly design engineers and few admin/support staff. Pete completes the design phase 3 months ahead of schedule, so he’s off to a good start! (Or is he?) Why Use TRIMS?

3. A Look Into The Future What if you had a crystal ball, and could see into the future? If Pete could do this, he would see that it is 5 years later, the APCM program is two years overdue, and a few hundred million dollars over budget. How did this happen? Pete’s “future self” explains what happened: “We finished the design phase ok, and then tried to manufacture some initial prototypes. We lost a couple of months trying to get the hang of producing plastic missile bodies, but the design engineers got pretty good at tweaking the machines every time they produced a missile shell. Because the plastic shells can’t really be bolted together, the design called for the two halves to be glued together with adhesive. Most of the prototypes worked well in testing, but it was kind of a pain to try to separate the two halves when a missile failed to launch, in order to try to figure out what was wrong. Still, most of the missiles worked, and we didn’t want to lose any more time, so we went ahead to production. That’s when the problems really started! We wanted an initial production run of 300 missiles, but when the contractor ran the line at full speed, the plastic forming machines would start to gum up after 25 missiles; they would jam, causing us to shut the entire line down while the machines were cleaned. And when the machines didn’t jam, we had problems manufacturing the shells themselves! Sometimes the shell would loosen itself from the vacuum assembly before it had completely cooled, leaving a slight indentation in the body. Some of these deformed shells didn’t have enough room for the electronics or the warheads to fit properly and the shell would have to be thrown out, or the body would be forced together anyway, damaging the electronics inside! Eventually we figured out what was happening, and started screening the shells before they went into assembly; but we’re still discarding 10% of the shells, which has slowed us down and increased the cost of each missile.” Why Use TRIMS?

4. A Look Into The Future “Things didn’t get any better when we deployed the missile. The same missiles which were 99% reliable in testing, only launched correctly 60% of the time in the field. And, since the missile bodies were sealed, the depots had no choice but to return them to us. After breaking several dozen of them open and inspecting them, we found lots of cracked solder joints and loose circuit cards. This didn’t make any sense at first since the contractor has a QA team that inspects every aspect of the missile as it’s being assembled, to make sure that every one that goes out is working perfectly. We finally got some failure-analysis experts in to diagnose the problem. That’s when we learned that out in the field, our missiles were being subjected to lots of bumps and jolts during transport as the vehicles they were in hit pot-holes, had to drive off-road, went over rubble, etc. When we asked the design engineers about this, they admitted that the subject of “rugged transport” hadn’t ever come up. They were able to modify the design so that the electronics are better protected during transport, but that made the missile a little bigger, so the production line had to be readjusted, and of course the new missiles no longer fit in the originally-designed shipping containers, so we have a warehouse of those just gathering dust...” Pete is jolted back to the present. “Hmmm... maybe I should revisit those decisions I just made.” Then a new thought strikes him: “If I missed these things, I wonder what else I’ve missed...” Why Use TRIMS?

5. In Summary 5 Years Later... • Program is 2 years behind schedule (and counting) • Program is $250+ Million over budget • Reasons (Testing): • Initial testing suffered delays because of new plastics technology • Difficult to analyze test failures • Reasons (Production): • Fabrication issues as a result of using unfamiliar technology • Delays caused by immature production process • Production errors result in increased delays, cost, waste • Production problems also create latent faults which surface during deployment • Reasons (Deployment): • Failures in the field because of rugged deployment conditions • Storage/Transport induced failures necessitated redesign Why Use TRIMS?

6. Some Things To Consider • Defense Science Board Studies Indicate that 80% of all Programs that Fail, Fail for Engineering not Technology Reasons • Example: N43 Reports that Over 65% of all Fleet Reliability Problems are Due to Inadequate Design • February 1998 GAO Report to the Subcommittee on Acquisition & Technology, Committee on Armed Services, U. S. Senate • Since 1975 Schedule Overruns Average 24% • Weapon Systems Overruns Cost 20% - 40% • March 2005 GAO Report: • Average Cost Increase Will be 28.5% in Only 4 Years • Reason: “Armed services order hardware with unproven technology, with too little time allowed for perfecting that technology before contractors attempt to insert it into weapons systems...” • With Acquisition Reform and Performance Specifications, We Must Monitor the Engineering Process to Understand Program Risks Do Not Get Technology Tunnel Vision Why Use TRIMS?

7. Why Process-Based Risk Management? • Cost overruns and schedule slippages are a result of technical issues or problems • Most technical issues are the result of engineering processes that either aren’t implemented, aren’t well understood, or are out-of-tolerance Schedule Cost Process Issues Technical Issues Time Proactive Reactive Managing Risk Using Cost & Schedule Data Is Like Driving Down The Highway With Your Eyes In The Rear-View Mirror Why Use TRIMS?

8. Evaluating Process Risk Permits More Timely & Efficient Mitigation T1 T2 T3 T4 T5 T6 Process Risks Product Risks Problems “A ‘Problem’ Is A Risk Whose Time Has Come” Why Use TRIMS?

9. Cost To Mitigate Increases Exponentially Design Test Production $100 Cost to Mitigate $10 $1 Time Why Use TRIMS?

10. Problems Cost To Mitigate Increases Exponentially Design Test Production Cost to Mitigate Product Risks Process Risks Time Risk Management or Damage Control? Why Use TRIMS?

11. What We Have Found • Continuous Insight of Systems Engineering Process is Key • Many Risk Management Actions are Problem Chasing Activities • Projects Become Schedule-Driven • People Become Over-Tasked • Communications Become Clogged • Product Quality Suffers • Configuration Management Becomes Impossible Why Use TRIMS?

12. What Is TRIMS? • TRIMS (Technical Risk Identification & Mitigation System): A Methodology for Doing Knowledge-Based, Process-Oriented Risk Management • Includes Set of Software Tools to Help Implement the Methodology (Navy Owned, Fixed Revision Cycle) • Outgrowth of DoD 4245.7-M and NAVSO P-6071 Templates • Fully Compatible with BMPCOE Combined Government / Industry Systems Engineering Model • Fully Supported: Training Classes, Computer-Based Training, On-line Help, 24-hour Help Desk Process Based And Predictive Why Use TRIMS?

13. TRIMS Proactive Methodology Risk Assessment Probability Effect • Guidance Documents • EngineeringProcess Model • Statistics Mission ProfileScheduleQualityCost PastExperience Planning &Tracking SystemArchitecture Risk Management • Milestone Dates TRIMS Risks PersonnelAssignments Next Actions QuestionWeighting • Risk Management • One Shots • Multiple Program Rollups • Automatic Updates Responsibilities Due Dates Action ItemTracking ComplianceStatus References Understanding The Future Effects Of Today's Decisions Why Use TRIMS?

14. TRIMS Proactive Methodology Risk Assessment “Risk” is a function of probability and effect. Determining these for your program is “Risk Assessment” Probability Effect • Guidance Documents • EngineeringProcess Model • Statistics Mission ProfileScheduleQualityCost PastExperience Planning &Tracking SystemArchitecture Risk Management • Milestone Dates TRIMS Risks PersonnelAssignments Next Actions QuestionWeighting • Risk Management • One Shots • Multiple Program Rollups • Automatic Updates Responsibilities Due Dates Action ItemTracking ComplianceStatus References Understanding The Future Effects Of Today's Decisions Why Use TRIMS?

15. TRIMS Proactive Methodology Risk Assessment Probability Effect • Guidance Documents • EngineeringProcess Model • Statistics Mission ProfileScheduleQualityCost PastExperience Planning &Tracking SystemArchitecture Risk Management To do “Risk Management” you also have to do planning and tracking: establish a plan to mitigate your risks, and then track it to ensure that the plan is being followed, and that it is having the desired effect! • Milestone Dates TRIMS Risks PersonnelAssignments Next Actions QuestionWeighting • Risk Management • One Shots • Multiple Program Rollups • Automatic Updates Responsibilities Due Dates Action ItemTracking ComplianceStatus References Understanding The Future Effects Of Today's Decisions Why Use TRIMS?

16. TRIMS Proactive Methodology Risk Assessment Probability Effect • Guidance Documents • EngineeringProcess Model • Statistics Mission ProfileScheduleQualityCost PastExperience Planning &Tracking SystemArchitecture TRIMS provides all of these capabilities, plus analysis, reporting, and workload-reduction features! Risk Management • Milestone Dates TRIMS Risks PersonnelAssignments Next Actions QuestionWeighting • Risk Management • One Shots • Multiple Program Rollups • Automatic Updates Responsibilities Due Dates Action ItemTracking ComplianceStatus References Understanding The Future Effects Of Today's Decisions Why Use TRIMS?

17. The Original “Willoughby” Templates PRODUCT TQM Money Phasing FUNDING DESIGN TEST PRODUCTION FACILITIES LOGISTICS MANAGEMENT • Integrated Test • Failure Reporting System • Uniform Test Report • Software Test • Design Limit • Life • Test, Analyze & Fix (TAAF) • Field Feedback • Modernization • Factory Improvements • Productivity Center • Logistics Support Analysis • Manpower & Personnel • Support & Test Equipment • Training Materials & Equipment • Spares • Technical Manuals • Manufacturing Strategy • Personnel Requirements • Data Requirements • Technical Risk Assessment • Production Breaks • Design Refer. Mission Profile • Design Requirements • Trade Studies • Design Policy • Design Process • Design Analysis • Parts & Mater. Selection • Software Design • Computer-Aided Design (CAD) • Design for Testing • Built-In Test (BIT) • Configuration Control • Design Reviews • Design Release • Manufacturing Plan • Qualify Manufacturing Process • Piece Part Control • Subcontractor Control • Defect Control • Tool Planning • Special Test Equipment (STE) • CAM • Manufacturing Screening TRANSITION PLAN Why Use TRIMS?

18. TRIMS Baseline Templates PRODUCT TQM Money Phasing CostAssessment FUNDING DESIGN TEST PRODUCTION FACILITIES LOGISTICS MANAGEMENT • Integrated Test • Failure Reporting System • Uniform Test Report • Software Test • Design Limit • Life • Test, Analyze & Fix (TAAF) • Field Feedback • TEMP Development/ Execution • Software Simulator • Modernization • Factory Improvements • Productivity Center • Field Visits/Site Surveys • Logistics Support Analysis • Manpower & Personnel • Support & Test Equipment • Training Materials & Equipment • Spares • Technical Manuals • Logistics Analysis Documentation • Manufacturing Strategy • Personnel Requirements • Data Requirements • Technical Risk Assessment • Production Breaks • Determining Defining Need for System • Prepare Requirement Documents • Quality Assurance • Design/Milestone Review Planning • Make or Buy Decisions • Technology Base Analysis • Schedule & Planning • Dim. Manuf. Sources & Mat. Shortage (DMSMS) • Design Refer. Mission Profile • Design Requirements • Trade Studies • Design Policy • Design Process • Design Analysis • Parts & Mater. Selection • Software Design • Computer-Aided Design (CAD) • Design for Testing • Built-In Test (BIT) • Configuration Control • Design Reviews • Design Release • Concept Studies & Analysis • Bread Board Development • Brass Board Development • Specification Dev/Allocation/ Validation • Design for Assembly • Prototype Development & Review • Manufacturing Plan • Qualify Manufacturing Process • Piece Part Control • Subcontractor Control • Defect Control • Tool Planning • Special Test Equipment (STE) • CAM • Manufacturing Screening • Production Fabrication • Environmental Issues New PMWS Templates TRANSITION PLAN Why Use TRIMS?

19. TRIMS Baseline Templates PRODUCT TQM The TRIMS SE Model includes the original 47 Templates from DoD 4245.7-M, plus another 21 templates that BMPCOE has added over the years – the principles have been around for 20 years, but the model is up-to-date! Money Phasing CostAssessment FUNDING DESIGN TEST PRODUCTION FACILITIES LOGISTICS MANAGEMENT • Integrated Test • Failure Reporting System • Uniform Test Report • Software Test • Design Limit • Life • Test, Analyze & Fix (TAAF) • Field Feedback • TEMP Development/ Execution • Software Simulator • Modernization • Factory Improvements • Productivity Center • Field Visits/Site Surveys • Logistics Support Analysis • Manpower & Personnel • Support & Test Equipment • Training Materials & Equipment • Spares • Technical Manuals • Logistics Analysis Documentation • Manufacturing Strategy • Personnel Requirements • Data Requirements • Technical Risk Assessment • Production Breaks • Determining Defining Need for System • Prepare Requirement Documents • Quality Assurance • Design/Milestone Review Planning • Make or Buy Decisions • Technology Base Analysis • Schedule & Planning • Dim. Manuf. Sources & Mat. Shortage (DMSMS) • Design Refer. Mission Profile • Design Requirements • Trade Studies • Design Policy • Design Process • Design Analysis • Parts & Mater. Selection • Software Design • Computer-Aided Design (CAD) • Design for Testing • Built-In Test (BIT) • Configuration Control • Design Reviews • Design Release • Concept Studies & Analysis • Bread Board Development • Brass Board Development • Specification Dev/Allocation/ Validation • Design for Assembly • Prototype Development & Review • Manufacturing Plan • Qualify Manufacturing Process • Piece Part Control • Subcontractor Control • Defect Control • Tool Planning • Special Test Equipment (STE) • CAM • Manufacturing Screening • Production Fabrication • Environmental Issues • CAM • Manufacturing Screening • Production Fabrication • Environmental Issues New PMWS Templates TRANSITION PLAN Why Use TRIMS?

20. Management Logistics Funding Design Test Production Facilities TransitionPlan TRIMS Tailoring User AddedCategories TRIMS Hierarchy 4.0 Design UserAddedTemplates 3 Tiers: Category Template Knowledge Base Question 4.6 Design Analysis UserAddedQuestions 4.6.1 Do the contractor's corporate standards identify design analysis as an integral part of the design process? Why Use TRIMS?

21. Tailoring TRIMS To Your Needs • Identify Risks Before they Become Problems • Build YOUR Knowledge into TRIMS • Use the Systems Engineering Knowledge Base as a Starting Point for A Good Engineering Process • Create a Concrete, Repeatable Process for Evaluating & Comparing Programs & Contractors • Ease the Learning Curve for New Employees • Retain Knowledge of Lost Employees • Share Knowledge • To Maximize Benefits of Using TRIMS • Apply as Early as Possible in Program Life • Apply to All Subcontractors, Divisions & Vendors Involved • Use Tool Regularly to Prioritize Resources & Make Decisions Why Use TRIMS?

22. Client Feedback Benefits • Reliability Established as a Performance Parameter • Manpower Shortfalls Identified • Numerous Best Practices Adopted • Engineering and Failure Analyses Implemented • Special Test Equipment and Test Assets Identified Early • Environmental Stress Screening Implemented • Concurrent Engineering Gains Obtained Problems • Not Easy to Implement (Socio-Politically) • Resistance to Change (Not Invented Here) • Process, Not Product Based • Few Actually Doing Concurrent Engineering Why Use TRIMS?

23. So How Does TRIMS Help? TRIMS SE Model Questions that would have helped Pete: • Eliminate the Design Reference Mission Profile (DRMP) 4.1.2 Has the contractor used detailed mission functional and environmental profiles to establish requirements and design margins for the system and its component parts? 4.1.4 Do environmental profiles cover all functional modes of operation including combat, transport, storage, handling, training, maintenance, and production? • Using a New Material for the Missile Body 4.5.1 Does the contractor's corporate policy include producibility as part of design reviews? 4.5.3 Will proof of manufacturing models be required prior to production? 4.5.7 Will effect of design on current manufacturing processes and facilities be assessed? 10.1.1 Will a manufacturing strategy be developed during the conceptual phase and included in solicitations for Engineering & Manufacturing Development (EMD)? 10.1.3 Will critical manufacturing technologies be identified and expertise acquired early for their development? • Design Contract: Mostly Design Engineers 4.5.2 Will manufacturing and producibility personnel be involved in the design process? 4.5.4 Will design and manufacturing engineers be collocated during development? 4.5.9 Will operations and customer support personnel be involved in the design process? 6.2.1 Will manufacturing processes be qualified at the prime contractor and major subcontractors? 6.2.5 Did the Production Readiness Review (PRR) indicate a stable design and manufacturing process? The design phase takes longer than originally planned, but the final design is easier to test, easier to produce, and better able to withstand the rigors of pre-launch deployment. Testing and Production proceed with only a few minor issues; when the cruise missile is finally deployed, it achieves a fielded reliability of 99.99% - the program is a complete success! Why Use TRIMS?

24. NO SURPRISES! Why Use TRIMS?

25. A Quick Look At TRIMS High level view gives an instant “snapshot” of your program! Why Use TRIMS?

26. A Quick Look At TRIMS Answering the “Expert Questions” helps determine risk levels; “Next Actions” helps mitigate process risks. Why Use TRIMS?

27. A Quick Look At TRIMS A variety of filtering options make it easy to review and analyze risk data. Why Use TRIMS?

28. A Quick Look At TRIMS Print reports at both the summary level... Why Use TRIMS?

29. A Quick Look At TRIMS ...and in more detail! Why Use TRIMS?

30. TRIMS - Total Risk Management P3RM Problem, Product & Process Risk Management, in a Single Tool! Why Use TRIMS?

31. Program Manager’s WorkStation • Reduce Risk Throughout Acquisition Process • KnowHow: What To Do • Database: Where To Get Help • TRIMS: Keep On Track • Tested and Used by Government and Industry • Taught at Defense Acquisition University and Industrial College of the Armed Forces • Accepted Tool/Methodology Within the DoD Acquisition Knowledge Sharing System • Acquisition Community Connection: • Program Management: http://acc.dau.mil/pm • Risk Management: http://acc.dau.mil/rm An Integrated Problem Solver Why Use TRIMS?

32. BMPCOE . . . Provides High ROI 1998 Winner Innovations In American Government 2000 Winner Hammer Award A place to obtain information about available resources to help make advances and improvements without making costly mistakes Call 1-800-789-4BMP www.bmpcoe.org Why Use TRIMS?

33. [ End of Presentation ]