1 / 24

Introduction to Machine Learning Multivariate Methods

Introduction to Machine Learning Multivariate Methods. 姓名 : 李政軒. Multiple measurements d inputs/features/attributes: d -variate N instances/observations/examples. Multivariate Data. Multivariate Parameters. Parameter Estimation. Estimation of Missing Values.

talasi
Download Presentation

Introduction to Machine Learning Multivariate Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Machine LearningMultivariate Methods 姓名:李政軒

  2. Multiple measurements d inputs/features/attributes: d-variate N instances/observations/examples Multivariate Data

  3. Multivariate Parameters

  4. Parameter Estimation

  5. Estimation of Missing Values What to do if certain instances have missing attributes Ignore those instances: not a good idea if the sample is small Use ‘missing’ as an attribute: may give information Imputation: Fill in the missing value Mean imputation: Use the most likely value Imputation by regression: Predict based on other attributes

  6. Multivariate Normal Distribution

  7. Mahalanobis distance: (x – μ)T∑–1(x – μ) measures the distance from x to μ in terms of ∑ (normalizes for difference in variances and correlations) Bivariate: d = 2 Multivariate Normal Distribution

  8. Bivariate Normal • Is probability contour plot of the bivariate normal distribution. • Its center is given by the mean, and its shape and orientation depend on the covariance matrix.

  9. If xi are independent, offdiagonals of ∑ are 0, Mahalanobis distance reduces to weighted (by 1/σi) Euclidean distance: If variances are also equal, reduces to Euclidean distance Independent Inputs: Naive Bayes

  10. Parametric Classification If p (x| Ci ) ~ N ( μi , ∑i ) Discriminant functions

  11. Estimation of Parameters

  12. Different Si Quadratic discriminant

  13. discriminant: P (C1|x ) = 0.5 likelihoods posterior for C1

  14. Shared common sample covariance S Discriminant reduces to which is a linear discriminant Common Covariance Matrix S

  15. Common Covariance Matrix S • Covariances may be arbitrary but shared by both classes.

  16. When xjj = 1,..d, are independent, ∑ is diagonal p (x|Ci) = ∏jp (xj |Ci) Classify based on weighted Euclidean distance (in sj units) to the nearest mean Diagonal S

  17. Diagonal S variances may be different

  18. Nearest mean classifier: Classify based on Euclidean distance to the nearest mean Each mean can be considered a prototype or template and this is template matching Diagonal S, equal variances

  19. Diagonal S, equal variances ? * • All classes have equal, diagonal covariance matrices of equal variances on both dimensions.

  20. As we increase complexity (less restricted S), bias decreases and variance increases Assume simple models (allow some bias) to control variance (regularization) Model Selection

  21. Different cases of the covariance matrices fitted to the same data lead to different boundaries.

  22. Binary features: if xj are independent (Naive Bayes’) the discriminant is linear Discrete Features Estimated parameters

  23. Multinomial (1-of-nj) features: xj Î {v1, v2,..., vnj} if xj are independent Discrete Features

  24. Multivariate linear model Multivariate polynomial model: Define new higher-order variables z1=x1, z2=x2, z3=x12, z4=x22, z5=x1x2 and use the linear model in this new z space Multivariate Regression

More Related