1 / 18

Decision Tree

Decision Tree. Developed by: Dr Eddie Ip Modified by: Dr Arif Ansari. Outline. Example Concept. Decision Tree: Example 1. Life insurance: whether preferred rate should be given? ID low risk group & give preferred rate Criteria: smoking ? overweight ?. Decision Tree: Example 2.

iirwin
Download Presentation

Decision Tree

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Decision Tree Developed by: Dr Eddie Ip Modified by: Dr Arif Ansari

  2. Outline • Example • Concept

  3. Decision Tree: Example 1 • Life insurance: whether preferred rate should be given? • ID low risk group & give preferred rate • Criteria: smoking ? overweight ?

  4. Decision Tree: Example 2 • Database of loan applications • Variable of interest = Loan approved / not approved (binary) • Predictors = age, gender, income group, own a house, …. • Similar application in Direct mail: To whom should I send mail ?

  5. Decision Tree: Concept • Classification of customers in DB known • Use historical data (“learn”) to guide your future decisions (classify)

  6. Steps • Build a model by “learning” from past data (learning/training set) • Tune model by using data not seen by model (testing set) • Evaluate accuracy of decision tree model by yet another new data set (evaluation set) • Use tree to classify new customers

  7. Decision Tree: Concept • Pattern recognition tool • Used in • recognizing hand writing • recognizing chemicals • recognizing ships at sea

  8. Decision Tree: Example • Vermont Country Store – student presentation

  9. Decision Tree: terminology • Variable of interest: response/ target (Y) • other variables : predictors (X) • loan example: X & Y • training set = records from DB

  10. Decision Tree: terminology • node (root & leaf) • child (left & right)

  11. Decision Tree: Concept • tree creates a set of bins into which records are tossed • start with root node (all records) • get best split so as to produce 2 homogeneous groups

  12. Decision Tree: Concept • Go down till a tree is formed • Stopping criteria: statistical test or grow-full-tree & prune

  13. Decision Tree: Technical issues • Example: loan application • measure of homogeneity/ diversity • e.g. Gini index, p(1-p)

  14. Decision Tree: concept • Grow tree = continue splitting • till no further split reduces diversity • Two philosophies: stopping rule or grow full tree & prune • Testing set may be required to stop growing or prune tree • In final decision tree, each terminal node is given a class

  15. Decision Tree: concept • NOTE: each terminal leaf node is not pure • Misclassification (error) rate =% incorrectly classified by tree

  16. Decision Tree: concept • Misclassification rate (sample) • Bigger tree ==> lower misclassifcation rate on training set • Big tree => overfit= “getting too close to data” • misclassification rate on training set over optimistic • More objective: evaluation set

  17. Decison Tree: products • CART = classification & regression tree(statistics) • C4.5/ C5.0 (machine-learning) • CHAID (statistics)

  18. Decision Tree : summary • method for classification & prediction • iterative splitting • training/testing to obtain an optimal tree • concept of overfit

More Related