Introduction To Data Mining

1. Introduction To Data Mining

2. What Is Data Mining? • A tool • Extraction of interesting (non-trivial, implicit, previously unknown and potentially useful) patterns or knowledge from huge amount of data • Core of KDD • Integration of Multiple technologies

3. Interpretation/ Evaluation Data Mining Preprocessing Patterns Selection Preprocessed Data Data Target Data Part of KDD(Knowledge Discovery in Databases) Knowledge adapted from: U. Fayyad, et al. (1995), “From Knowledge Discovery to Data Mining: An Overview,” Advances in Knowledge Discovery and Data Mining, U. Fayyad et al. (Eds.), AAAI/MIT Press

4. Integration of Multiple Technologies Artificial Intelligence Machine Learning Database Management Statistics Visualization Algorithms Data Mining Other knowledge

5. Why Data Mining? • We are drowning in data (Data explosion problem ), but starving for knowledge! • Solution: Data warehousing and data mining • Data warehousing and on-line analytical processing • Mining interesting knowledge (rules, regularities, patterns, constraints) from data in large databases • A lot of potential applications • Market analysis and management • Target marketing, customer relationship management (CRM), market basket analysis, cross selling, market segmentation • Risk analysis and management • Forecasting, customer retention, improved underwriting, quality control, competitive analysis • Health care …

6. Data mining process State the problem + hypothesis Knowledge-base

7. Knowledge from Data Mining • Association rules • Sequential Association • Classification rules • Clustering • Deviation Detection • …

8. Identify association in the data: (correlation [A,B] and causality[A->B]) Indicate significance of each association (only interesting if its confidence exceed a certain measure) Not all the Association is interesting (too trivial, negative association) E.g. market-basket analysis “Find groups of items commonly purchased together” People who purchase fish are likely to purchase wine Association Rules

9. Find event sequences that are unusually likely Requires “training” event list, known “interesting” events Must be robust in the face of additional “noise” events Uses: Failure analysis and prediction Technologies: Dynamic programming (Dynamic time warping) “Custom” algorithms “Find common sequences of warnings/faults within 10 minute periods” Warn 2 on Switch C preceded by Fault 21 on Switch B Fault 17 on any switch preceded by Warn 2 on any switch Sequential Associations

10. Classify a set of data based on their values in certain attributes Requires “training data”: have predefined attributes Uses: Profiling Technologies: Generate decision trees (results are human understandable) Neural Nets “Route documents to most likely interested parties” English or non-english? Domestic or Foreign? Classification rules

11. Group a set of data base on the conceptual clustering principle(i.e. maximizing the intraclass similarity and minimizing the interclass similarity) No “training data”: Without predefined attributes Uses: Demographic analysis Technologies: Self-Organizing Maps Probability Densities Conceptual Clustering “Group people with similar travel profiles” George, Patricia Jeff, Evelyn, Chris Rob Clustering

12. Find unexpected values, outliers Uses: Failure analysis Anomaly discovery for analysis Technologies: clustering/classification methods Statistical techniques visualization “Find unusual occurrences in IBM stock prices” Deviation Detection

13. Popular Data Mining Techniques • Supervised • Decision trees • Rule induction • Regression models • Neural Networks … • Unsupervised —K-means clustering —Self organized maps …

14. Supervised vs. Unsupervised • Supervised algorithms » Learning by example: – Use training data which the value of the response variable is already known – Create a model by running the algorithm on the training data – Identify a class label for the incoming new data » Driven by a real business problems and historical data • Unsupervised algorithms » Do not use training data. » Patterns may not be known in advance

15. Supervised Algorithms

16. Decision Trees • A tree structure where non-terminal nodes represent tests on one or more attributes and terminal nodes reflect decision outcomes. • Advantages of decision trees —Understandable —Relatively fast —Easy to translate into SQL queries • Disadvantages of decision trees — Limited to one output attribute — Decision tree algorithms are not so stable • Types of decision trees —CHAID: Chi-Square Automatic Interaction Detection —CART: Classification and Regression Trees …

18. Figure 1.1 A decision tree for the data in Table 1.1

20. The extraction of useful independent if-then rules from data based on statistical significance If rules cause prediction confliction -> solve it according to confidence Advantage and disadvantage —Understandable —not cover all the possible situation E.g. IF Swollen Glands = Yes THEN Diagnosis = Strep Throat IF Swollen Glands = No & Fever = Yes THEN Diagnosis = Cold IF Swollen Glands = No & Fever = No THEN Diagnosis = Allergy IF = Antecedent THEN = Consequence Rule induction

21. Neural Networks • Non-linear predictive models that learn through training and resemble biological neural networks in structure • Means of efficiently modeling large and complex problems in which there may be hundreds of predictor variables that have many interactions • Disadvantage • Difficult understand • Can require significant amounts of time to train, to prepare data • …

22. Figure 2.2 A multilayer fully connected neural network

23. Regression Models • Statistical techniques • Using existing values to forecast what other values will be. Y = a + b1(X1) + b2(X2) + b3(X3) + b4(X4) + b5(X5) … • A lot of types regression (linear regression, logistic regression …)

24. K-Means Clustering • Unsupervised algorithm • Steps of algorithm • Choose a value for K, the total number of clusters. • Randomly choose K points as cluster centers. • Assign the remaining instances to their closest cluster center. • Calculate a new cluster center for each cluster. • Repeat steps 3-5 until the cluster centers do not change.

26. A Hypothesis for the Credit Card Promotion Database A combination of one or more of the dataset attributes differentiate Acme Credit Card Company card holders who have taken advantage of the life insurance promotion and those card holders who have chosen not to participate in the promotional offer.

27. Figure 2.3 An unsupervised cluster of the credit card database

28. Choosing a Data Mining Technique • Know which kind knowledge you want to get • Know your data --What is the interaction between input and output attributes? --What is the Distribution of the Data? --Which Attributes Best Define the Data? • Know the difference among different data mining techniques

29. Questions to Determine Data Mining Applicability • Can the problem be clearly defined? • Does potentially meaningful data exist? • Does data contain hidden knowledge or is it just filled with facts? • Is the “juice worth the squeeze?”

30. Discovery-based (deductive process) Mine data warehouse and others Can provide information you didn’t expect Verification-based (inductive process) DSS tool for data warehouse Pre-defined queries Data Mining vs. OLAP

31. For hidden knowledge Try to get the answer as accurate as possible Results are the analysis of the data Data need to be prepare before producing results For specific question Answer to query is 100% accurate if data correct Results are subset of data Need not prepare data Data Mining vs. Data Query