1 / 26

Image Recognition using Hierarchical Temporal Memory

Image Recognition using Hierarchical Temporal Memory. Radoslav Škoviera Ústav merania SAV Fakulta matematiky, fyziky a informatiky UK. Image Recognition. Applications: Digital image databases, surveillance, industry, medicine

lynton
Download Presentation

Image Recognition using Hierarchical Temporal Memory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Image Recognition using Hierarchical Temporal Memory Radoslav Škoviera Ústav merania SAV Fakulta matematiky, fyziky a informatiky UK

  2. Image Recognition • Applications: Digital image databases, surveillance, industry, medicine • Tasks: Object recognition, automatic annotation, content based image search • Input: Digital Image • Single object • Scene (multiple objects – clutter, occlusion, merging) • Output: Description of the input image • Keywords, scene semantics, similar images • Subtasks: image segmentation, feature extraction, classification

  3. Motivation • Image recognition • Very easy for us humans (and [other] animals) • Computers can‘t do it neither quickly, nor accurately enough, yet • Good motivation for the researchers in the field of AI – bio-inspired models

  4. Hierarchical Temporal Memory (HTM) • Developed by Jeff Hawkins and Dileep George (Numenta) • Hierarchical tree-shaped network • Bio-inspired – based on large scale model of the neocortex • Consists of basic operational units – nodes • Each node uses the same two-stage learning algorithm: 1) Spatial Learning (Pooling) 2) Temporal Learning (Pooling) • Learning is performed layer-by-layer • Nodes have receptive fields – each (except for the top node) can look only at a portion of the input image

  5. Spatial Learning • Observe common patterns in the input space (training images) • Group them into clusters of spatially simillar patterns • Use only one representative of each cluster • Generate „codebook“ • Input space and spatial noise reduction

  6. Temporal Learning • Uses time sequences to learn correlations of spatial patterns

  7. Temporal Learning

  8. Temporal Learning

  9. Temporal Learning • In each training step, TAM is increased at the locations corresponding with the co-occurring codebook patterns according to the update function defined as follows:

  10. Inference & Classification • Uses simlar dataflow as learning • Two stages of inference in each node: • Spatial inference – find the closest pattern in the codebook • Temporal inference – calculate membership into temporal groups • Classification – HTM itself does not classify images, it only transforms input space into another (hopefully more inviariant) space • External classifier must be used

  11. ATM Security • ATM (automatic teller machine) semiatomatic fraud detection system • Detection of masked individuals interacting with the ATM through the ATM‘s camera – possibility of illegal activity • Pilot system implemented and tested in an experimental environment • Using Kinect as an input device

  12. Kinect • RGB camera developed for the XBOX game console • Capable of providing depth image for the scene and a „skeleton“ if a person is detected on the scene

  13. Experiment Setup

  14. Face Image Segmentation using Kinect

  15. Face Image Segmentation using Kinect • Two image classes: normal and anomalous faces

  16. ATM Security – Results • Image set inflated with translated, rotated and mirrored copies of the original images • k-NN classifier in the input space was compared with the combination of the HTM and k-NN and HTM and SVM classifier • Scenario 1: The whole data set was used and • Scenario 2: Translated images were excluded from the training set

  17. New features and algorithmsfor the HTM • New temporal pooler • Images transformed to different image spaces • different image features • Various settings for the temporal pooler • SOM as spatial pooler

  18. Testing of new image features • Dataset: selected images from Caltech 256 • 10 classes, 30 testing and 30 training images per class • Single layer network • With 1-NN classifier as top node • Image features extracted from image patches corresponding to the receptive fields of nodes

  19. Results

  20. problems - background

  21. problems - background

  22. Thank you for your attention

More Related