Intro to Neural Networks

1. Supervised Learning: Perceptrons and Backpropagation Intro to Neural Networks

2. Neural Network == • Connectionist /ism== • Parallel Distributed Processing (PDP)

3. Neural Networks assume • Intelligence is emergent

4. 1943 - McCullough Pitts Artificial Neuron

5. 1943 - McCullough Pitts Artificial Neuron

6. Perceptron Learning 1958

7. Perceptron Learning 1958

8. Perceptron Learning 1958

9. Perceptron Learning 1958

10. Perceptron Learning 1958

11. Linear Seperability Problem 1965

12. Linear Seperability Problem 1965

13. Backpropagation • Used to train multilayer feedforward networks

14. Backpropagation

15. Backpropagation • Used to train multilayer feedforward networks • Assumes a continuous activation function

16. Backpropagation - Activation

17. Backpropagation • Used to train multilayer feedforward networks • Assumes a continuous activation function • Delta rule

18. Backpropagation Delta rule • Perceptron update rule was: • Backprop update rule is:

19. Backpropagation Delta rule • Error of an output node:

20. Backpropagation Delta rule • Error of a hidden node:

21. Backpropagation Delta rule

22. Backpropagation Delta rule

23. Backpropagation Delta rule

24. Backpropagation Delta rule

25. Backpropagation • demo

26. Inductive Bias

27. Inductive Bias • Encoding / Feature Extraction • # neurons used • # layers used • Output mapping

28. Domains • Classification

29. Domains • Classification • Pattern Recognition

30. Domains • Classification • Pattern Recognition • Content Addressable Memory

31. Domains • Classification • Pattern Recognition • Content Addressable Memory • Prediction

32. Domains • Classification • Pattern Recognition • Content Addressable Memory • Prediction • Optimization

33. Domains • Classification • Pattern Recognition • Content Addressable Memory • Prediction • Optimization • Filtering

34. The good • Degrade gracefully

35. The good • Degrade gracefully • Solve ill-defined problems

36. The good • Degrade gracefully • Solve ill-defined problems • Flexible

37. The good • Degrade gracefully • Solve ill-defined problems • Flexible • Generalization

38. The bad • Time & Memory

39. The bad • Time & Memory • Black box

40. The bad • Time & Memory • Black box • Trial and Error

41. When not to use Feedforward net • If you can draw a flow chart or formula

42. When not to use Feedforward net • If you can draw a flow chart or formula • If a piece of hardware or software already exists that does what you want

43. When not to use Feedforward net • If you can draw a flow chart or formula • If a piece of hardware or software already exists that does what you want • If you want to functionality to evolve

44. When not to use Feedforward net • If you can draw a flow chart or formula • If a piece of hardware or software already exists that does what you want • If you want to functionality to evolve • Precise answers are required

45. When not to use Feedforward net • If you can draw a flow chart or formula • If a piece of hardware or software already exists that does what you want • If you want to functionality to evolve • Precise answers are required • The problem could be described in a lookup table

46. When to use feedforward net • You can define a correct answer

47. When to use feedforward net • You can define a correct answer • You have a lot of training data with examples of right and wrong answers

48. When to use feedforward net • You can define a correct answer • You have a lot of training data with examples of right and wrong answers • You have lots of data but can’t figure how to map it to output

49. When to use feedforward net • You can define a correct answer • You have a lot of training data with examples of right and wrong answers • You have lots of data but can’t figure how to map it to output • The problem is complex but solvable

50. When to use feedforward net • You can define a correct answer • You have a lot of training data with examples of right and wrong answers • You have lots of data but can’t figure how to map it to output • The problem is complex but solvable • The solution is fuzzy or might change slightly