A Statistical Anomaly Detection Technique based on Three Different Network Features

1. A Statistical Anomaly Detection Technique based on Three Different Network Features Yuji Waizumi Tohoku Univ.

2. Background • The Internet has entered the business world • Need to protect information and systems from hackers and attacks • Network security has been becoming important issue • Many intrusion/attack detection methods has been proposed

3. Intrusion Detection System • Two major detection principles: • Signature Detection • Attempts to flag behavior that is close to some previously defined pattern signature of a known intrusion • Anomaly Detection • Attempts to quantify the usual or acceptable behavior and flags other irregular behavior as potentially intrusive.

4. Motivation • Anomaly detection system • Pro: can detect unknown attacks • Con: many false positives • Improve the performance of Anomaly detection system • Analyze the characteristics of attacks • Propose method to construct features as numerical values from network traffic • Construct detection system using the features

5. Classification of Attacks • DARPA Intrusion Detection Evaluation • DoS: Denial of Service • Probe: Surveillance of Targets • Remote to Local(R2L), User to Root(U2R): Unauthorized Access to a Host or Super User

6. Re-classification of Attacks • Classification by Traffic Characteristics • DoS, Probe • Traffic Quantity • Access Range • Probe • Structure of Communication Flows • DoS, R2L, U2R • Contents of Communications To detect attacks with above characteristics, it is necessary to construct features corresponding those classes.

7. Network Traffic Feature • Numerical values(vectors) expressing state of traffic • We propose three different network feature sets • Based of re-classification of attacks • Analyzed independently

8. Time Slot Feature (34 dimension) • Count various packets, flags, transmission and reception bytes, and port variety by a unit time • Estimate scale and range of attacks • Target • Probe (Scan) • DoS • Each slot is expressed as a vector Ex) (TCP,icmp,SYN,FIN,RST,UDP,DNS,…)

9. rst flag (port 23) rst flag (port 21) ftp scan telnet scan Examples (Time Slot Feature) Element value Vector element Values are regularizes as mean=0, variance=1.0 normal traffic only

10. Flow Counting Feature • Flow is specified by (srcIP, dstIP, srcPort,dstPort,protocol) • Count packets, flags, transmission and reception bytes in a flow • Target • Scan with illegal flags • Ports used as backdoors • TCP:19 dim. , UDP:7 dim.

11. Decrease of SYN packet Port sweep(scan) Examples (Flow Counting Feature) Specific packets of attacks are extremelyhigh and low. Element value Vector element Normal traffic

12. Flow Payload Feature • Represent content of communication • Histogram of character codes of a flow • Count 8bit-unit(256 class) • Transmission and reception are counted independently (total 512 class) • Target • Buffer overflow • Malicious code

13. Examples (Flow Payload Feature) Specific character of attacks are extremelyhigh and low. Normal traffic imap attack

14. Modeling Normal Behavior • Each packet appears based on protocol Correlations between elements of the feature vectors • Profile based on correlations can represent normal behavior of network traffic

15. Principal Component Analysis:PCA • Extract correlation among samples as Principal Component • Principal Component lay along sample distribution Non-correlated data Principal Component

16. Principal Component Projection Distance Anomaly sample Discriminant Function • Projection Distance • Long Distant Samples: • Unordinary traffic • Break Correlation Detection Criterion

17. Detection Algorithm • Independent Detection • The three features are used for PCA independently • "Logical OR" operation for detection alerts by each feature Features Alert PCA Time Slot OR Network Traffic Alert PCA Flow Counting Alert Alert PCA Flow Payload

18. Performance Evaluation • Two Examine Scenario • Scenario1 • Learn Week1 and 3 • Test Week4 and 5 • Scenario2 • Learn Week 4 and 5 • Test Week 4 and 5 • More Practical Situation • Real network traffic may include attack traffic • Criterion for Evaluation • Detection rate when number of miss-detection (false positive) per day is 10

19. Data Set • Data Set • 1999 DARPA off-line intrusion detection evaluation test set • Contain 5 weeks data (from Monday to Friday) • Week1,3: Normal traffic only • Week2: Including attacks (for learning) • Week4,5: Including attacks (for testing)

20. Scenario 1 Result 2003 2000

21. Scenario 2 Result • NETAD • Use IP address as white list • Overfit learning data • Proposed Method • Independent of IP address • Evaluate only anomaly of traffic

22. Detection Results every Features Scenario 1 Low detection overlap （TS） （FC） （FP） Time Slot Feature(TS) 22 9 5 Flow Counting Feature(FC) 13 6 Each feature detect different characteristic attacks Flow Payload Feature(FP) 44 （TS） & （FC） & （FP） 5 Scenario 2 （TS） （FC） （FP） Time Slot Feature（TS） 37 7 2 # of Detection by both TS & FP Flow Counting（FC） 8 3 Flow Payload(FP） 40 （TS） & （FC） & （FP） 3 # of Detection by FP only # of Detection by all Three Features

23. Conclusion • For network security • Classification attacks into three types • Construct three features corresponding to three attack characteristics • Detection method with PCA • Learning the three features independently • Higher detection accuracy • With samples including attacks