Dealing with the Differences in Hurricane Models

1. Dealing with the Differences in Hurricane Models Catastrophe Risk Management Seminar October 7 & 8, 2002 Ronald T. Kozlowski Martin M. Simons William Gardner

2. Kozlowski - ASOP #38 - Using Models Outside the Actuary’s Area of Expertise Simons - Why are hurricane models different? Gardner - Quantification of model uncertainty and variations in model results Agenda

3. 11-15-95 American Academy of Actuaries Casualty Practice Council Meeting ASB Casualty CouncilWhy is a standard needed: Regulators and other users . . .question the applicability of these analyses when they are based in large part on the results of models that are either outside of the normal range of actuarial work or for which key parts of the model contain proprietary information and are not subject to normal disclosure processes. ASOP #38 - Development Background

4. ASB Casualty Council believes that a new standard on the Use of Complex Models in Actuarial Practices is required. . . . Address the level of understanding of the model and any underlying theories . . . Also address what control processes must be used by the actuary in establishing a reliance on the model’s output. 1996 - ASB Task Force on Complex Models Adopted by the Actuarial Standards Board June 2000 ASOP #38 - Development Background

5. Karen F. Terry, Chair Kay A. Cleary Alice H. Gannon Paul E. Kinson Ronald T. Kozlowski Godfrey Perrott David A. Lalonde Jeffrey F. McCarty Daniel M. Scheibenreif A. Eric Thorlacius Joan M. Weiss Kurt Reichle ASB Task Force on Complex Models

6. Issues • Is a standard needed? • Should it address when to use a model? • Who should standard apply to? • Raising the bar • Reliance on experts • Proprietary issues

7. Section 1 - Purpose, Scope, Cross References, and Effective Date Section 2 - Definitions Section 3 - Analysis of Issues and Recommended Practices Section 4 - Communications & Disclosures Appendices Using Models Outside the Actuary’s Area of Expertise (P/C)

8. This standard applies to actuaries who use models that incorporate specialized knowledge outside of the actuary’s own area of expertise when performing professional services in connection with property and casualty insurance coverages . . . This standard applies to all models whether or not they are proprietary in nature. The standard is intended to be used in conjunction with other actuarial standards of practice (ASOPs) providing guidance for actuarial work, . . . If a conflict exists between this standard and applicable law, compliance with applicable law is not considered to be a deviation from this standard. Section 1 - Scope

9. When using a modelthat incorporates specialized knowledge outside the actuary’s own area of expertise,the actuary should: Determine appropriate reliance on experts Have basic understanding of the model Evaluate whether model is appropriate for intended application Confirm appropriate validation has occurred Determine appropriate use of model Level of effort in understanding and evaluating should be consistent with intended use and its materiality to results of actuarial analysis Section 3.1 - Introduction

10. Actuary should consider whether: individual is an expert model has been reviewed by experts in field there are standards that apply to the model or to the testing and validation of the model . . . certified as having met such standards Section 3.2 - Appropriate Reliance on Experts

11. “The actuary should be familiar with the basic components of the model and understand both the user input and the model output . . .” Model Components (familiar with basic components, basic understanding, is it generally accepted, how tested or validated, level of independent expert review) User Input (detail required to produce results consistent with intended use) Model Output (consistent with intended use) Section 3.3 - Understanding of the Model

12. Is the model appropriate for particular actuarial analysis? May consider: Applicability of historical data Developments in relevant fields.Impossible to remain up to date if not an expert in the field Section 3.4 - Appropriateness of the Model for the Intended Application

13. The actuary should evaluate the user input and reasonableness of the model output Directed to ASOP 23, Data Quality Reasonability may consider Results of alternative models or methods Historical observations Consistency of relationships Sensitivity to variations in assumptions Section 3.5 - Appropriate Validation

14. Having completed sections 3.2-3.5, the actuary should use judgment to determine whether it is appropriate to use the model results, making any compensating adjustments as necessary. Section 3.6 - Appropriate Use of the Model

15. An actuary may rely on another actuary who followed this standard Satisfied that other actuary’s evaluation performed in accordance and is appropriate for intended purpose Disclose such reliance Section 3.7 - Reliance on Model Evaluation by Another Actuary

16. HURRICANE MODELS WHY ARE THEIR RESULTS DIFFERENT?

17. PRIMARY COMPONENTS Meteorology Vulnerability Actuarial SECONDARY COMPONENTS Statistical Computer MODEL COMPONENTS

18. Hurricane Frequencies geographical frequencies frequency by magnitude directional frequencies Hurricane Tracks initial position initial direction directional movements METEOROLOGICAL COMPONENTS

19. Hurricane Characteristics central pressure wind speed forward velocity radius of maximum winds radius of hurricane force winds far field pressure other characteristics Rankine vortex Beta parameter METEOROLOGICAL COMPONENTS - CONTINUED

20. Surface water temperature Geographical impacts Distance from landfall Surface roughness topography buildings trees METEOROLOGICAL COMPONENTS - CONTINUED

21. Data used to derive damage functions Insurance claim data Post hurricane inspections Wind tunnel tests Engineering judgment Data used to verify damage functions Insurance claim data Post hurricane inspections VULNERABILITY CHARACTERISTICS

22. Adjustments to damage functions New building codes Mitigation measures Individual structural characteristics Roof type Cladding type Structure height Many others VULNERABILITY CHARACTERISTICS - CONTINUED

23. Policy provisions Policy limits Deductibles Coinsurance Insurer practices What constitutes a covered claim? What constitutes a total loss? ACTUARIAL COMPONENTS

24. VERIFICATION statistically reasonable results? UNCERTAINTY Causes of the variations? SENSITIVITY How does each component impact result? STATISTICAL TESTS

25. Verification of reasonable results Chi-square goodness-of-fit test Kolmogorov-Smirnov test Cramer-von Mises test Sensitivity and Uncertainty Latin Hypercube Sampling STATISTICAL TESTS - CONTINUED

26. Incomplete knowledge Myriad of variables Very short historical period Constantly changing environment Structures Population shifts Policy variations Insurer practice variations SO WHY DO MODEL RESULTS DIFFER?