Evaluation Methods for Internet Security Technology (EMIST)

1. Evaluation Methods for Internet Security Technology (EMIST) NSF Cyber Trust PI Meeting and DETER workshop Newport Beach, CA, Sept. 2005

2. EMIST TEAM • PSU: G. Kesidis**(PI), P. Liu†, P. McDaniel, D. Miller • UCD: K. Levitt (PI), F. Wu*, J. Rowe, C.-N. Chua • ICSI: V. Paxson* (PI), N. Weaver* • Purdue: S. Fahmy (PI), N. Shroff, E. Spafford • SPARTA: D. Sterne (PI), S. Schwab*, R. Ostrenga, R. Thomas, S. Murphy, R. Mundy • SRI: P. Porras, L. Breismeister • **overall PI, *expt lead/co-lead, †EMIST ESVT lead • PMs: Joe Evans (NSF) and Douglas Maughan (DHS) • Sister project: DETER cyber security testbed

3. Outline • Team. • Goals. • Publications. • Tools released. • Talks for DETER workshop Wed 09/28/05. • Y3 activities.

4. EMIST goals • Develop scientifically rigorous testing frameworks and methodologies for defenses against attacks on network infrastructure: scale-down with fidelity. • Develop experiments to yield deeper understanding of how previous attacks have, and future attacks will, affect the Internet and its users. • Develop prototypical experiments (benchmarks) and associated databases of: • topologies and topology generators • attack and background traffic traces and generators • defenses • special-purpose devices (meters, virtual nodes, etc.) • metrics for scale-down fidelity, performance, overhead, etc.

5. EMIST goals (cont) • Consult in the build-out of the DETER testbed and demonstrate its usefulness to vendors, researchers and customers of defense technology. • Allow for open, convenient, rigorous, unbiased and secure testing of cyber defenses on DETER in order to expedite their commercial deployment. • Quickly and publicly disseminate our results.

6. 2004 EMIST publications • N. Weaver, I. Hamadeh, G. Kesidis and V. Paxson, “Preliminary results using scale-down to explore worm dynamics”, in Proc.  ACM WORM, Washington, DC, Oct. 29, 2004. • P. Porras, L. Biesemeister, K. Levitt, J. Rowe, K. Skinner, A. Ting, “A hybrid quarantine defense”, in Proc. ACM WORM, Washington, DC, Oct. 29, 2004. • S.T. Teoh, K. Zhang, S.-M. Tseng, K.-L. Ma and S. F. Wu, “Combining visual and automated data mining for near-real-time anomaly detection and analysis in BGP”, in Proc.ACM VizSEC/CMSEC-04, Washington, DC, Oct. 29, 2004.

7. 2005 EMIST publications • A. Kumar, N. Weaver and V. Paxson, "Exploiting Underlying Structure for Detailed Reconstruction of an Internet-scale Event", in Proc. ACM IMC 2005. • R. Pang, M. Allman, M. Bennett, J. Lee, V. Paxson, B. Tierney, "A First Look at Modern Enterprise Traffic ", in Proc. ACM IMC 2005. • S. Schwab, B. Wilson, R. Thomas, “Methodologies and Metrics for the Testing and Analysis of Distributed Denial of Service Attacks and Defenses,” MILCOM, Atlantic City, NJ, Oct. 2005. • L. Li, S. Jiwasurat, P. Liu, G. Kesidis, Emulation of Single Packet UDP Scanning Worms in Large Enterprises, In Proc. 19  International Teletraffic Congress (ITC-19), Beijing, Aug. 2005. • Q. Gu, P. Liu, C.-H. Chu, Hacking Techniques in Wired Networks, In The Handbook of Information Security, Hossein Bidgoli et al. (eds.), John Wiley & Sons. • S. Sellke, N. B. Shroff, and S. Bagchi, "Modeling and AutomatedContainment of Worms", In Proceedings of the International Conference in Dependable Systems and Networks (DSN), June 2005. • R. Chertov, S. Fahmy, and N. B. Shroff, "Emulation versusSimulation: A Case Study of TCP-Targeted Denial of Service Attacks",Purdue University Technical Report, September 2005. • L. Briesemeister and P. Porras. Microscopic simulation of agroup defense strategy. In Proceedings of Workshop on Principles of Advanced and Distributed Simulation (PADS), pages 254-261, June 2005. • C. H. Tseng, T. Song, P. Balasubramanyam, C. Ko, and K. Levitt, "A Specification-based Intrusion Detection Model for OLSR“, in Proc. RAID, Sept. 2005.

8. 2005 EMIST publications • K. Zhang, S. Teoh, S. Tseng, R. Limprasittipom, C. Chuah, K. Ma, andS.F. Wu. PERFORMING BGP EXPERIMENTS ON A SEMI-RELISTIC INTERNET TESTBEDENVIRONMENT. in the 2nd International Workshop on Security inDistributed Systems (SDCS), conjunction with ICDCS, 2005. • W. Huang, J. Cong, C. Wu, F. Zhao, and S.F. Wu. DESIGN, IMPLEMENTATION,AND EVALUATION OF FRITRACE. in 20th IFIP International InformationSecurity Conference, May, 2005, Chiba, Japan, Kluwer AcademicPublishers. • G. Hong, F. Wong, S.F. Wu, B. Lilja, T.Y. Jansson, H. Johnson, and A.Nilsson. TCPTRANSFORM: PROPERTY-ORIENTED TCP TRAFFIC TRANSFORMATION.in GI/IEEE SIG SIDAR Conference on Detection of Intrusions and Malware& Vulnerability Assessment (DIMVA), Vienna, Austria, July, 2005, LNCS,Springer. • J. Crandall, S.F. Wu, and F. Chong. EXPERIENCES USING MINOS AS A TOOLFOR CAPTURING AND ANALYZING NOVEL WORMS FOR UNKNOWN VULNERABILITIES. inGI/IEEE SIG SIDAR Conference on Detection of Intrusions and Malware &Vulnerability Assessment (DIMVA), Vienna, Austria, July, 2005, LNCS,Springer. • G.H. Hong and S.F. Wu. ON INTERACTIVE INTERNET TRAFFIC REPLAY. in the8th Symposium on Recent Advanced Intrusion Detection (RAID), Seattle,September, 2005, LNCS, Springer. • J. Crandall, Z. Su, S.F. Wu, and F. Chong. ON DERIVING UNKNOWNVULNERABILITIES FROM ZERO-DAY POLYMORPHIC & METAMORPHIC WORM EXPLOITS.To appear in 12th ACM Conference on Computer & Communication Security(CCS’2005), Alexandria, November 7-11, 2005.

9. EMIST tools • EMIST Experiment Specification and Visualization Tool (ESVT) 2.0 released in May ’05 with: • more advanced traffic viz features including link data and SQL interface, and • ability to import output from a scale-free topology generator (with associated plotting tool). • Offline netflow audit tool released in May ’05. • Online Scriptable Event System (SES) and, data analysis measurement tools. • XML worm configuration and worm modeling. • TCPOpera traffic generator and ELISHA viz tool. • BGP topology capture tool. • Experimental technical reports.

10. ICSI worm demo: source models for testing net-based detectors • We are developing layer 4 (TCP/UDP) “source models”. • Process of representing normal systems: • Derived from traces of a medium-scale enterprise (10K hosts) • Store traffic information in database • Classify host types & application sessions based on measurements • Create background traffic by sampling hosts and sessions • Near-term goal is to mimic the Layer 4 behavior of normal hosts • Testing against Approximate TRW worm containment • Overlay worm traffic by adding worm-functionality to models • Longer term goals: • investigate *abstract* source models • apply to other containment technology

11. UC Davis / SRI worm demo: collaborative host-based defense • Hosts that are not protected by network defenses can protect themselves from worm attack by collaborating with collections of other hosts to exchange alerts. • A preliminary end-host collaborative worm defense exchanging failed connection reports will be demonstrated: • with respect to its ability to protect against worm spread • in the presence of realistic background traffic. • A 2000 virtual node experiment that uses our two tools: • the NTGC traffic generator and • the UCD Worm Emulator

12. SPARTA DDoS demo • FloodWatch defense deployed on both PCs andCloudShield appliances, as well as Juniper routers. • A range of data collection and EVST visualization tools will be explored. • The theme is examination of the experimental methodology, in particular: • the degree to which accurate detection and response characteristics can be calculated versus • the limited fidelity of generated background traffic.

13. Purdue: Method and Tools for High-Fidelity Emulation of DoS Attacks • Simulation versus emulation of DoS attack experiments are compared. • As a case study, we considered low-rate TCP-targeted DoS attacks. • Specific measurement-fidelity issues of the DETER testbed were resolved. • We found that software routers such as Click provide a flexible experimental platform, but require detailed understanding of the underlying network device drivers to ensure they are correctly used. • We also found that an analytical model and ns-2 simulations closely match with typical values of attack pulse lengths and router buffer sizes.

14. UCD: Requirements and Toolsfor Routing Experiments • Tools: Requirements and Design (with SPARTA) • ER (Entity Relationship) Information Visualization • Experiments: • Interaction of BGP/OSPF/P2P • Cross-layer routing dynamics/interactions • Per-Update OASC Experiment • Analysis of address ownership • DDoS/Routing Interaction (with Purdue) • DDoS impacts on BGP

15. PSU BGP demo:Large-Scale eBGP Simulator (LSEB) • Our goal is large Internet-scale (global) routing attack modeling and measurement. • Methodology: • intial AS topologies drawn from PREDICT Routeviews • 20k java threads running across DETER hosts • simulate all BGP message level interactions • maintain route tables for all reachable prefixes • Future work:  • realistic AS forwarding delay models  • modeling iBGP  • scale-down of experiments with more complex/realistic BGP speakers • defense deployment and evaluation on DETER

16. PSU ESVT demo • ESVT rendering of UDP/TCP worm emulation in an enterprise:   • We have emulated SQL slammer on a 1000 node enterprise network and compared the realism achieved by VM (jail), real LANs, and virtual nodes.   • We are currently emulating TCP Blaster worm considering issues including the fidelity of our Blaster modeling technique, and the impact of background traffic.   • Note that no defense is involved, just a local block of dark addresses used for detection.

17. Y3 Activities • Release of reusable code developed for on-going attack/defense experiments, in particular: • ESVT 3.0+ with integrated trace audit tool, spectral analysis, etc. • Synthesize background traffic analogous to trace datain DETER experiments on same topology. • BGP ESVT. • Continued outreach, in particular BGP ESVT components to the ops community. • Collaborate with DETER on, e.g., experimental workbench (SEW), RIB output collection.

18. Y3 Activities (cont) • For each attack experiment, a summary document that described in particular: • Experimental methodologies. • Metrics for experimental realism in defense evaluation. • Benchmark attack experiments for specific classes of defenses. • Experimental Tech Reports: • Experiment archiving and repeatability issues. • Critical assessments of all items in deterlab’s experimenters’ tools web pages. • Summer 2006 attack/defense demonstration experiments.