Download
1 / 72
750 likes | 952 Views
Online Robust Dictionary Learning. Cewu Lu, Jianping Shi, and Jiaya Jia The Chinese University of Hong Kong. Dictionary Learning. Denoise [ Mairal et al. 2008]. Upsampling [ Couzinie-Devy 2010]. Image Classification [Wang et al. 2010]. Background Subtraction
E N D
Online Robust Dictionary Learning Cewu Lu, Jianping Shi, and JiayaJia The Chinese University of Hong Kong
Dictionary Learning Denoise [Mairal et al. 2008] Upsampling[Couzinie-Devy 2010] Image Classification [Wang et al. 2010] Background Subtraction [Cong et al. 2010]
Dictionary Learning Let be a set of signal. Let be a set of “basis vectors”.
Dictionary Learning is “adapted” to if it can represent with a few basis vector. Spare
Robust Dictionary Learning L2 norm data fitting is not a robust measure. A toy example: X={2,5,6,9,10,12,14,15,18}
Robust Dictionary Learning L2 norm data fitting is not a robust measure. A toy example: Outliers X={2,5,6,9,10,12,14,15,80000} Inliers
Robust Dictionary Learning [ Wagner et al 2009], [ Wang et al 2012], [ Zhao et al 2011 ] L1 norm is a robust measure. A toy example: X={2,5,6,9,10,12,14,15,80000}
Robust Dictionary Learning [ Wagner et al 2009], [ Wang et al 2012], [ Zhao et al 2011 ] L1 norm is a robust measure. A toy example: X={2,5,6,9,10,12,14,15,80000}
Robust Dictionary Learning Outliers Incorrect Dictionary Inliers Non-Robust Dictionary Learning
Robust Dictionary Learning Outliers Correct Dictionary Inliers Robust Dictionary Learning
Robust Dictionary Learning [ Wagner et al 2009],[ Wang et al 2012], [ Zhao et al 2011 ] But, it is not widely used…. Why? Outliers Dictionary Inliers Non-Robust Dictionary Learning
Online Dictionary Learning Because…. L1 norm data-fitting hasn’t closed-form heavy computation Large-scale data Dynamic data We need Online.
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data Current
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data History Current
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data History Current
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data Current History
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data Current History
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data Current History
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data Current History
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Data Current History
Online Dictionary Learning Online Solver [Mairal et al 2010]: Dictionary Update Dictionary Data History Current
Online Dictionary Learning Online Solver [Mairal et al 2010]:
Online Robust Dictionary Learning Our goal: Make robust dictionary learning online. Less Computation Less Memory
Online Robust Dictionary Learning Dictionary Update online Forget history Data History Current Robust Require Whole Data Outliers Dictionary Inliers
Online Robust Dictionary Learning Dictionary Update online Forget history Data History Challenging Current Robust Require Whole Data Outliers Dictionary Inliers
Online Robust Dictionary Learning • Our Online Approach (Online) • Robustness Analysis (Robust) • Discussion
Online Robust Dictionary Learning • Our Online Approach • Robustness Analysis • Discussion
Online Dictionary Learning Settings: Each min-batch data contains h data point. We have two parameter matrixes and . … … Data (Min-batch)
Online Dictionary Learning Initialization: and are zero matrixes. Dictionary D is a random matrix. Update … … Data Current
Online Dictionary Learning General Framework Update … … Data History Current
Online Dictionary Learning General Framework Update … … Data History Current
Online Dictionary Learning General Framework Update … … Data Current History
Online Dictionary Learning General Framework Update … … Data Current History
Online Dictionary Learning General Framework Update … … Data Current History
Online Dictionary Learning General Framework Update … … Dictionary Data History Current
Our Online Approach In step: Previous Dictionary … … Data t Min-batch Data: History Current Sparse code:
Our Online Approach Solve Current Dictionary Data t Min-batch Data: Current History Sparse code:
Our Online Approach New Data Information History Information Solve Current Dictionary Data t Min-batch Data: Current History Sparse code:
Record and only. Our Online Approach
Online Robust Dictionary Learning • Our Online Approach • Robustness Analysis • Discussion
Our Online Approach Solve Current Dictionary Data Min-batch Data: Current History Sparse code:
Robustness Analysis Solve
Robustness Analysis (Proof) Solve Iterative Reweighted Least Squares
Robustness Analysis (Proof) Solve Solve
Robustness Analysis (Proof) Solve Solve
Robustness Analysis History data New data
