1 / 42

Non-linear hypotheses

Non-linear hypotheses. Neural Networks: Representation. Machine Learning. Non-linear Classification. x 2. x 1. size. # bedrooms. # floors. age. But the camera sees this:. What is this?. You see this: . Computer Vision: Car detection. Not a car. Cars. Testing : What is this? .

orenda
Download Presentation

Non-linear hypotheses

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Non-linear hypotheses Neural Networks: Representation Machine Learning

  2. Non-linear Classification x2 x1 size # bedrooms # floors age

  3. But the camera sees this: What is this? You see this:

  4. Computer Vision: Car detection Not a car Cars Testing: What is this?

  5. pixel 1 Learning Algorithm pixel 2 Raw image pixel 2 Cars “Non”-Cars pixel 1

  6. pixel 1 Learning Algorithm pixel 2 Raw image pixel 2 Cars “Non”-Cars pixel 1

  7. pixel 1 Learning Algorithm 50 x 50 pixel images→ 2500 pixels (7500 if RGB) pixel 2 Raw image pixel 2 Cars pixel 1 intensity “Non”-Cars pixel 2 intensity pixel 2500 intensity Quadratic features ( ): ≈3 million features pixel 1

  8. Neurons and the brain Neural Networks: Representation Machine Learning

  9. Neural Networks Origins: Algorithms that try to mimic the brain. Was very widely used in 80s and early 90s; popularity diminished in late 90s. Recent resurgence: State-of-the-art technique for many applications

  10. The “one learning algorithm” hypothesis Auditory Cortex Auditory cortex learns to see [Roe et al., 1992] [Roe et al., 1992]

  11. The “one learning algorithm” hypothesis Somatosensory Cortex Somatosensory cortex learns to see [Metin & Frost, 1989]

  12. Sensor representations in the brain Human echolocation (sonar) Seeing with your tongue Haptic belt: Direction sense Implanting a 3rd eye [BrainPort; Welsh & Blasch, 1997; Nagel et al., 2005; Constantine-Paton & Law, 2009]

  13. Neural Networks: Representation Model representation I Machine Learning

  14. Neuron in the brain

  15. Neurons in the brain [Credit: US National Institutes of Health, National Institute on Aging]

  16. Neuron model: Logistic unit Sigmoid (logistic) activation function.

  17. Neural Network Layer 1 Layer 2 Layer 3

  18. Neural Network “activation” of unit in layer matrix of weights controlling function mapping from layer to layer If network has units in layer , units in layer , then will be of dimension .

  19. Model representation II Neural Networks: Representation Machine Learning

  20. Forward propagation: Vectorized implementation Add .

  21. Neural Network learning its own features Layer 1 Layer 2 Layer 3

  22. Other network architectures Layer 1 Layer 2 Layer 3 Layer 4

  23. Examples and intuitions I Neural Networks: Representation Machine Learning

  24. Non-linear classification example: XOR/XNOR , are binary (0 or 1). x2 x2 x1 x1

  25. Simple example: AND 1.0

  26. Example: OR function -10 20 20

  27. Examples and intuitions II Neural Networks: Representation Machine Learning

  28. Negation:

  29. Putting it together: -10 -30 10 20 20 -20 20 20 -20

  30. Neural Network intuition Layer 1 Layer 2 Layer 3 Layer 4

  31. Handwritten digit classification [Courtesy of YannLeCun]

  32. Handwritten digit classification [Courtesy of YannLeCun]

  33. Multi-class classification Neural Networks: Representation Machine Learning

  34. Multiple output units: One-vs-all. Pedestrian Car Motorcycle Truck Want , , , etc. when pedestrian when car when motorcycle

  35. Multiple output units: One-vs-all. Want , , , etc. when pedestrian when car when motorcycle Training set: one of , , , pedestrian car motorcycle truck

More Related