Learning from Labeled and Unlabeled Data using Graph Mincuts

1. Learning from Labeled and Unlabeled Data using Graph Mincuts Avrim Blum and Shuchi Chawla May 24, 2001

2. Utilizing unlabeled data • Cheap and available in large amounts • Gives no obvious information about classification • Gives information about distribution of examples • Useful with a prior • Our prior: ‘close’ examples have a similar classification

3. + - Mincut Classification using Graph Mincut

4. Why not nearest neighbor?

5. Why not nearest neighbor? Classification by 1-nearest neighbor

6. Why not nearest neighbor? Classification by Graph Mincut

7. Self-consistent classification • Mincut minimizes leave-one-out cross validation error of nearest neighbor • May not be the best classification • But, theoretically interesting!

8. Assigning edge weights • Several approaches: • Decreasing function in distance eg. Exponential decrease with appropriate slope • Unit weights but connect only ‘nearby’ nodes How near is ‘near’? • Connect every node to k-nearest nodes What is a good value of k? • Need an appropriate distance metric

9. How near is ‘near’? • All pairs within  distance are connected • Need a method of finding a ‘good’  • As  increases, cut value increases • Cut value = 0  supposedly no-error situation (Mincut- 0)

10. Mincut- 0 does not allow for noise in the dataset • Allow longer distance dependencies • Grow  till the graph becomes sufficiently well connected • Growing till the largest component contains half the nodes seems to work well (Mincut- ½ )

11. Other ‘hacks’ • Weigh edges to labeled and unlabeled examples differently • Weigh different attributes differently eg. Use information gain as in decision trees • Weigh edges to positive and negative example differently: for a more balanced cut • Use mincut value as an indicator of performance

12. Some results