Cost Risk Analysis

1. Mort Anvari, CRB Office Cost Risk Analysis Mort.Anvari@us.Army.mil

2. Outline • Definition of Risk and Uncertainty Analysis • Methods of Risk Analysis • Risk Facets • Schedule Risk

3. Risk Assessment Definition Purpose The purpose of a Risk Assessment is to provide Management and decision makers with a realistic range of possible outcomes, given a fixed set of baseline assumptions! Process • Quantify the impact of the different Risk factors on cost • Identify inter-relationships between these Risk factors • Combine individual Risk factors into an overall estimate range • Interpret and report results

4. Risk and Uncertainty Definitions Risk is the Occurrence of an Outcome Subject to a Known Pattern of Random Variation, i.e. Known-Unknown. ( Changes in those Technical Parameters that are Captured in the Estimating Methodology ) Uncertainty is the Occurrence of an Outcome Subject to Unknown Random Fluctuations, i.e. Unknown-Unknown. ( Changes in those Parameters that are not Part of the Estimating Methodology )

5. Cost Risk & Uncertainty Definition Cost Risk and Uncertainty Analysis is Placing Probability Distribution Around a Point Estimate Cost Risk Analysis Assumes that the Analyst has Correctly Identified the Factors (Parameters) that Influence Cost and Taken them into Account when Producing the Point Estimate If the Point Estimate is not a function of Influencing Factors, Performing a Risk Analysis will be of Little Value

6. Risk and Uncertainty Analysis Risk Analysis is a Requirement All Cost Estimates for Major Defense Acquisitions Must be Accompanied by a Formal Risk Analysis “Defense Acquisition Management Policies and Procedure” DoD 5000.2

7. Reasons for Doing Risk Analysis Point Estimates are Almost Always Wrong, because Changes in the Specifications Changes in the Requirements Optimistic Estimates of Advances in Technology Funding Instability Schedule Slippage

8. Reasons for Risk

9. Point Estimate Method for Producing the Point Estimate Must be the Same as that for Estimating the Risk In fact, the Point Estimate for Total Cost Should be a By-Product of the Cost Risk Analysis Analyst Should not Obtain a Point Estimate and then Later Apply a Totally Different Method to Assess the Risk

10. Risk Reduction Cost Risk Analysis will not Reduce the Risk Inherent in a Program Risk Analysis Help Program Managers Understand the Nature of the Risk Involved, and the Uncertainty Associated with Cost Estimates Risk Analysis result in a more Realistic Assessment of the Funding Required and Likelihood of Exceeding the Point Estimate

11. Technical Schedule Programmatic Cost Estimating Cost Risk Facets

12. High Risk Low Risk Moderate Risk Probability of Occurrence Increasing Low Risk Severity of Consequence Risk Concepts

13. When to do Risk Analysis Risk analysis reduces the uncertainty between requirements and funding

14. Risk Analysis Methods Qualitative Methods (e.g., Subjective Assessments of Low, Medium, or High Risk) are of Most Use when There is Little or no Historical Data Available or Firm Requirements Have not Yet been Established Quantitative Methods are Considered where Probability Distribution on Cost Elements or Drivers can be Estimated from Historical Data or Deduced from Expert Opinion

15. Quantitative Methods Analytical Method involves the Mathematical Determination of a Total Cost Distribution from its Components Cost Distributions Simulation Method Involves the Computer Generation of Random Costs from Component Distributions and Aggregation into a Total Cost Distribution

16. Other Methods Prediction Intervals, like stating “There is X% Chance that total cost will fall between Y and Z” Predication Intervals are too gross a summary measure of the distribution of total cost They are not needed if the entire distribution can be estimated

17. Risk Analysis Approaches • Detailed Network & Stochastic • Discrete Technical, Schedule, and Estimating Risks • Detailed Monte Carlo Simulation (each WBS) Effort • Bottom Line Monte Carlo Simulation • Add a Risk Factor/Percentage Detail

18. CER: Use Regression Analysis to Relate Cost Drivers to Historical Costs • WBS: Apply Probability Distribution to Cost Elements and Aggregate them into a total cost • Stochastic: Break Down the Acquisition and Management Processes into their Components Activities and Events in Proper Time Sequence Networks Ways to Perform Risk Analysis

19. Total Cost WBS Cost Distribution PDF Frequency P.E. Sum PE MLC (Mode) + • = CDF Confidence 50% P.E. + 20% Sum PE MLC P.E. +Many More Probability Distributions

20. ? + + = Cost Tech Schedule P.E. PDF P.E. + + P.E. P.E. + CDF + P.E. P.E. + Many More + Many More Combining Risk Distributions

21. Probability Density Function

22. Cumulative Distribution Functions

23. Coeff. of Variability 0.18 Range Minimum $1,318 Range Maximum $3,466 Range Width $2,148 Mean Std. Error $5.45 Statistical Results Forecast: Statistic Value Trials 5,000 Mean $2,195 Median (approx.) $2,147 Mode (approx.) $2,100 Standard Deviation $385 Variance $148,391 Skewness 0.47 Kurtosis 2.79 Mean = first moment Variance = second moment Std. Deviation = (Variance)1/2 Skewness = third moment Kurtosis = fourth moment

24. Total Forecast Results in FY05 $M 0% $1,242 10% $1,719 20% $1,842 30% $1,947 40% $2,043 50% $2,135 60% $2,231 70% $2,359 80% $2,507 90% $2,709 100% $3,389 Percentiles Cumulative Probability Not to Exceed

25. Schedule Risk Schedule Risk

26. Cost Schedule Curve Fixed Cost Technology Outdate Parallel Effort More ECP Less Mature Design Life Cycle Cost Typical Min Optimal Development Schedule

27. Schedule Goals • Commercial Drivers • Technology Drives Schedule • Constraint Schedule • Goal is ROI Maximization Defense Cost • DoD Drivers • Funding Drives Schedule • Unconstrained Schedule • Goal is Cost Reduction Commercial Development Schedule

28. DoD Program Schedule Drivers Funding Allocation Cost Acquisition Process Program Execution Cycle Time Reduction Goal Development Schedule

29. Development Schedule Mean Cycle Time to IOC DoD Pre-1992 Starts 132 months (11 years) Post-1992 Starts 89 months (7.4 years) on-going F-22 216 months (18 years) IOC 2004 Comanche 264 months (22 years) IOC 2006 Commercial Boeing 777 54 months (4.5 Years)

30. TOTAL ENGINE IN-HOUSE Example of R&D Funding Profile F Y 0 8 F Y 0 7 F Y 9 5 F Y 9 6 F Y 9 7 F Y 9 8 F Y 9 9 F Y 0 0 F Y 0 1 F Y 0 2 F Y 0 3 F Y 0 4 F Y 0 5 F Y 0 6 F Y 0 9

31. Historical Perspective 78% of Programs had less than 10% Schedule Slippage 92% of Programs BAFO had Raised no Issue with the Government Proposed Schedule • Program Managers’ Priority • Superior Performance • Low Acquisition Cost • Low Operation Cost • Shortened Schedule Shortening the Cycle Time Provides Opportunity for Cost Reduction

32. Reduced Cycle Time Opportunity Business Case AS-IS TO-BE Program 1 Program 1 Program 4 Program 2 Program 3 Cost Program 2 Program 5 Program 4 Program 5 Program 6 Program 3 Program 6 x x/2 x/2 Cycle Time Result From: - Starting Programs too Soon - Starting Too many Programs - Inefficient Acquisition Process Reduced Cycle Time will Result In: - Fielding of Newer Technology - Lower Cost per Project - Reduced Exposure to Annual Funding Reduction

33. Reasons for Risk

34. Information Organization, Structure, and Mapping • Expert Identification and Communication • Interdependency of Risk Issues • Impact of Global Factors (e.g., Industrial Base) • Modeling • Probability Distribution Assumptions • Packaging Range Estimate • Interpretation of the Results • Risk Mitigation Plan • PPBES and Point Estimates Cost Risk Analysis Challenges

35. Summary • Funding Allocation Puts more Constraint on the Program Schedules than Acquisition Process and Program Execution • Shortening the Cycle Time Provides Cost Reduction Opportunity with Greater Operational Capability • Cost Schedule Risk Analysis Tools and Techniques Should be Utilized to Demonstrate the Effect of Schedule Trade-off

36. At the end of the day, the cost analyst ends up being the system architect-integrator-engineer as the details are worked through.

37. Input Process Output Model Supports Inputs (Descriptions) Process (Models) Outputs (Costs) Requirements Acquisition Assumptions Design Parameters Risk Assessment ASARC IT OIPT CAIG, DAB APB PPBES AOA / EA Cost Analysis Process Databases, Tools, & Models $ Types of Cost Analysis Estimate Known Solving For: How Much Does the System Cost? System Descriptions, Cost Models System Costs OLD Approach Historical System Description and Costs How to Cost a System? Cost Models / CER System Descriptions, Design & What System Can I Get For My Dollars? NEW Cost Targets Models Approach Performance Parameters Initiated Development of Performance Based and Design Based Cost Models 37