1 / 27

MIW METOC Metrics: Ops Analysis

MIW METOC Metrics: Ops Analysis. Tom Murphree and David Meyer Naval Postgraduate School (NPS) murphree@nps.edu Bruce Ford and Manuel Avila Clear Science, Inc. (CSI) bruce@clearscienceinc.com Paul Vodola, Matt McNamara, Luke Piepkorn, and Ed Weitzner Systems Planning and Analysis (SPA)

ahmed-baxter
Download Presentation

MIW METOC Metrics: Ops Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MIW METOC Metrics: Ops Analysis Tom Murphree and David Meyer Naval Postgraduate School (NPS) murphree@nps.edu Bruce Ford and Manuel Avila Clear Science, Inc. (CSI) bruce@clearscienceinc.com Paul Vodola, Matt McNamara, Luke Piepkorn, and Ed Weitzner Systems Planning and Analysis (SPA) pvodola@spa.com Presented at MIW METOC Metrics Symposium III CNMOC, Stennis Space Center, MS 1 February 2008

  2. MIW METOC Metrics: Ops Analysis • Outline • Previous Ops Analysis Modeling • Weather Impact Assessment Tool (WIAT) • Linking METOC and Customer Performance • Role of Ops Analysis in the MIW Metrics Process MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  3. MIW METOC Metrics: Ops Analysis • WIAT – Overview • WIAT is • A pc-based, stochastic simulation tool used for assessing the impacts of METOC forecasts and METOC phenomena on strike operations • WIAT can • Simulate modern naval strike operations with a focus on fixed strike (STK) and kill box interdiction (KI) and close air-support (CAS) • Take as input both actual weather conditions and forecasts of conditions with varying lead times and accuracies • Emulate actual mission planning processes days and hours ahead of mission execution • Model customer decisions and outcomes based on weather predictions and other factors • Allows users to: • Simulate impacts of METOC forecasts and phenomena on spectrum of operational situations • Assess mission effectiveness employing a variety of METOC support situations • Determine the payoff/ROI for increased forecasting accuracy MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  4. 24% Observed and Forecasted Conditions Category % Weather Cancellations MIW METOC Metrics: Ops Analysis WIAT – Overview From NPS thesis research of LCDR Jake Hinz, USN, 2004 For official use only MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  5. MIW METOC Metrics: Ops Analysis • WIAT – Time-Error Forecast Analysis • Metrics collected through the online real-time collection system can be used as input to the ops analysis models to show the link between METOC performance and customer performance • Purpose of the following parametric analysis is to show that improved timing of forecasted features leads to improved operational effectiveness in Strike warfare • Perfect forecasting case is compared with timing errors of one and three days MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  6. 0 1 3 0 1 3 Time Shift (Days) MIW METOC Metrics: Ops Analysis METOC PERFORMANCE METRICS WIAT – Time-Error Forecast Analysis • Large amount of weather near the RYG thresholds decreases FAC & POD • Higher accuracy rate in forecasting Green Wx than Yellow or Red • Three-day errors not significantly worse than the one-day error MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  7. 0 1 3 0 1 2 3 Time Shift (Days) MIW METOC Metrics: Ops Analysis OPERATIONALMETRICS WIAT – Time-Error Forecast Analysis • Effectiveness per sortie is degraded due to inaccurate forecasting • Results in 11% decrease in targets killed per mission • Sorties with degraded BDA increased from 42 to 68 • Increase from 16% of all missions flown to 27% MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  8. Correlating METOC Support & Customer Performance 49% 42% Strike Mission Efficiency 39% 27% 23% Operational Performance 16% % Sorties with No BDA 40% 52% 100% Forecast Accuracy MIW METOC Metrics: Ops Analysis WIAT – Time-Error Forecast Analysis MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  9. Correlating METOC Support & Customer Performance 49% 42% Strike Mission Efficiency 39% • Where does the METOC community need to fall on the X-axis? • How can we make this determination? • NMETLS • NMAWC Metrics • Others? 27% 23% Operational Performance 16% % Sorties with No BDA 40% 52% 100% Forecast Accuracy MIW METOC Metrics: Ops Analysis WIAT – Time-Error Forecast Analysis MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  10. MIW METOC Metrics: Ops Analysis • Role in MIW Metrics • Develop and apply a set of operational analysis and modeling tools for: • Modeling MIW operations, especially METOC inputs to planning process and impacts on operational outcomes • Evaluating operational implications of METOC metrics based on real world data • Developing quantitative operational thresholds for the performance of METOC products • Identifying gaps and excesses in METOC support, and recommend mitigation strategies (DOTMLP-F) • Assessing alternative CONOPS for METOC support • Informing budgetary decisions at directorate and resource sponsor levels • Support initial operational use of tool set to provide assessment of METOC impact on customer performance MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  11. MIW METOC Metrics: Ops Analysis • Application • Tool allows for assessment of the impacts of METOC support products and services across a wide range of MIW scenarios • Determine sensitivities to increased Accuracy • Effect of enhanced Timeliness Scenario Multiple combinations of METOC products and services support MIW METOC Product MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  12. MIW METOC Metrics: Ops Analysis • Application • Use collected data to determine performance relative to the determined sensitivities • Answers the question “HOW WELL ARE WE DOING?” MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  13. MIW METOC Metrics: Ops Analysis • Application • For each Scenario – METOC Product pair, determine current performance in relation to the defined benchmarks • Identify gaps & excesses • Provide strategy recommendations (DOTMLP-F) MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  14. MIW METOC Metrics: Ops Analysis • Application Overview Periodic re-evaluation provides a feedback loop between collected data & performance benchmarks MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  15. MIW METOC Metrics: Ops Analysis QUESTIONS? MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  16. MIW METOC Metrics: Ops Analysis • Outline • Role of Ops Analysis in the MIW Metrics Process • Previous Ops Analysis Modeling • Weather Impact Assessment Tool (WIAT) • Linking METOC and Customer Performance MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  17. MIW METOC Metrics: Ops Analysis • Notional Tool • Notional MIW Weather Impact Assessment Tool • Simulates modern Mine Warfare operations • Inputs both actual environmental conditions and predictions with varying lead times and accuracies • Emulates actual mission planning processes days and hours ahead of mission execution • Models customer decisions and operational outcomes based on environmental predictions and other factors • Value • Simulates impacts of METOC predictions and phenomena on spectrum of MIW situations • Facilitates determination of phenomena which have significant impact on MIW operations • Evaluates MIW effectiveness employing a variety of METOC support situations MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  18. MIW METOC Metrics: Ops Analysis • Outline • Role of Ops Analysis in the MIW Metrics Process • Previous Ops Analysis Modeling • Weather Impact Assessment Tool (WIAT) • Linking METOC and Customer Performance MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  19. MIW METOC Metrics: Ops Analysis WIAT – Role in METOC Metrics Currently, WIAT has been used to experiment with various levels of METOC support to STW MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  20. Visibility MIW METOC Metrics: Ops Analysis WIAT – METOC Features • Forecasted and actual conditions over time • 0.5o spatial resolution • Temporal resolution: varying, down to 2 hrs • METOC features modeled • Cloud coverage (three layers) • Precipitation • Visibility • Wind speed • Includes inputs for other forecasted and observed conditions • Weather features impact all components of the simulations • Threat & friendly forces • Weapon probability of kill • Sensor probability of detection • Allows for the input of various forecast accuracies to capture operational effects of forecast uncertainty MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  21. 0 1 2 3 0 1 3 Time Shift (Days) Time Shift (Days) MIW METOC Metrics: Ops Analysis METOC IMPACTMETRICS WIAT – Time-Error Forecast Analysis • Mission change accuracy decreases by 40 – 60 percent • Over 30 fewer missions are saved when forecast timing errors are introduced MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  22. MIW METOC Metrics: Ops Analysis • Model Development Approach • Develop realistic vignettes and scenarios for MIW, including mission-level metrics of success • Identify the environmental factors that most significantly influence MIW platform performance and operational effectiveness • Describe and model the flow of METOC information from reachback, through in-theater support, to the MIW commander • Describe and model the MIW mission planning process, to include how and when METOC information is incorporated • Develop operational model(s) for MIW • Assess METOC support performance and impact on customer performance MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  23. ATO DRIVERS TACAIR Missions METOC Impacts on Operational Metrics Threat Forces Forecasted Weather • Changes Made During Planning Process Due to Weather Forecasts • Changes Made During Mission Due to Forecasts and Actual Weather • Mission Success • Opportunity Costs of Decisions Based on Forecasts and Actual Weather ATO Actual Weather Blue Ground Forces ISR Assets MIW METOC Metrics: Ops Analysis WIAT – Model Schematic MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  24. Resource (Sortie) Allocation Initial Plan Generation JPITL METOC Forecasting Products Strike Missions CAS Missions KI Missions Daily Target List ATO Mission (#AC, Weapons) ATO Mission (#AC, Weapons) ATO Mission (#AC, Weapons) ISR, FACs MIW METOC Metrics: Ops Analysis WIAT – Mission Planning Process Flight Plan Generation and Modification Timeline Flight Ops Execution Forecast Sortie Generation Planning Forecast • Secondary modificationsand sortie cancellations • Primary mission modifications • Initial flight plan generation Multiple Flight Plan Updates Over Time due to Forecasting & ISR MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  25. MIW METOC Metrics: Ops Analysis WIAT – Output Metrics • WIAT contains an automated output processing tool • WIAT provides an automated link between METOC support and operational performance WIAT Captures METOC Metrics… …and Operational Impact Metrics MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  26. MIW METOC Metrics: Ops Analysis • Application • Based on these determinations, METOC support benchmarks can be developed to establish goals and evaluate real world performance MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

  27. MIW METOC Metrics: Ops Analysis Linking METOC to Customer Performance WIAT – Time-Error Forecast Analysis • Forecast Accuracy with Three-day errors not significantly worse than the One-day error • Decrease in Forecast Accuracy leads to an increase in sorties that could not perform BDA due to Wx 0 1 3 Time Shift (Days) MIW METOC Metrics Ops Analysis, Feb 08, pvodola@spa.com

More Related