Chapter 7 Supervised Hebbian Learning

1. Chapter 7 Supervised Hebbian Learning

2. Outline • Linear Associator • The Hebb Rule • Pseudoinverse Rule • Application

3. Linear Associator

4. Hebb’s Postulate “When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.” D. O. Hebb, 1949 B A

5. Hebb Rule(1/2)

6. Hebb Rule(2/2)

7. Batch Operation

8. Performance Analysis(1/2) Input patterns are orthonormal,

9. Performance Analysis(2/2)

10. Example(orthonormal)

11. Example(not orthogonal)

12. Example(not orthogonal)

13. PseudoinverseRule(1/3)

14. PseudoinverseRule(2/3)

15. Pseudoinverse Rule(3/3) is Moore-Penrose Pseudoinverse. The Pseudoinverse of a real matrix P is the uniqui matrix that satisfies

16. Relationship to the Hebb Rule

17. Relationship to the Hebb Rule

18. Example

19. Autoassociative Memory

20. Tests 50% Occluded 67% Occluded Noisy Patterns (7 pixels)

21. Variations of Hebbian Learning Basic Rule: Learning Rate: Unsupervised: Smoothing: Delta Rule:

22. Solved Problems

23. Solved Problems

24. Solved Problems Solution:

25. Solved Problems

26. Solved Problems

27. Solved Problems