1 / 19

Hopfield s Network

What is Hopfield's Network?. This is network capable of associating its input with one of patterns stored in network's memoryNext logical questionsHow patterns are stored in memory?How inputs are supplied to the networkWHAT IS THE TOPOLOGY OF THE NETWORK. Inputs. The inputs of the Hopfield netw

yama
Download Presentation

Hopfield s Network

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Hopfields Network Dragoljub Pokrajac 2003

    2. What is Hopfields Network? This is network capable of associating its input with one of patterns stored in networks memory Next logical questions How patterns are stored in memory? How inputs are supplied to the network WHAT IS THE TOPOLOGY OF THE NETWORK

    3. Inputs The inputs of the Hopfield network are values x1,,xN -1? xi?1 Hence, the vector x=[x1 xN] represents a point from a hyper-cube

    4. Topology

    5. A Neuron of Hopfields Network

    6. How Neuron is Characterized Neuron is characterized by its state si The output of the neuron is the function of the neurons state: yi=f(si) The applied function f is soft limiter which effectively limits the output to the [-1,1] range

    7. Operations of Neuron Neuron initialization When an input vector x arrives to the network, the state of i-th neuron, i=1,,N is initialized by the value of the i-th input: si=xi

    8. Updates of Neurons Subsequently While there is any change:

    9. Convergence The subsequent computation of the network will occur until the network does not stabilize The network will stabilize when all the states of the neurons stay the same IMPORTANT PROPERTY: Hopfields network will ALWAYS stabilize after finite time

    10. What is Output of the Network Output of the network is vector y=[y1yn] consisting of neuron outputs when the network stabilizes

    11. Why Hopfields Network is Important? Assume that we want to memorize M different N-dimensional vectors What does it mean to memorize? It means: if a vector similar to one of memorized vectors is brought to the input of the Hopfield network The stored vector closest to it will appear at the output of the network

    12. The following can be proven. If the number M of memorized N-dimensional vectors is smaller than N/4ln(N) Then we can set the weights of the network as: Where W contains weights of the network a symmetric matrix with zeros on main diagonal NONE of the neurons is connected to itself Such that the vectors correspond to the stable states of the network

    13. In Other Words If vector xm* is on the input of the Hopfields network the same vector xm* will be on its output If a vector close to vector xm* is on the input of the Hopfields network The vector xm* will be on its output

    14. Hence The Hopfield network memorizes by embedding knowledge into its weights

    15. Why Hopfields Network Works With the N neurons, total number of output vectors is 2N Only a small fraction (M) of these vectors represent useful information (one of M stored vectors) This make the Hopfield network highly redundant NOTE: Hopfield network resembles Error Correcting Codes (ECOC)

    16. Main Problems with Hopfield Network What is close OK, the output associated to input is one of stored vectors closest to the input However, the notion of closeness is hard encoded in the weight matrix and we cannot have influence on it

    17. Main Problems with Hopfield Network Spurious states Assume that we memorized M different patterns into a Hopfield network The network may have more than M stable states Hence (the Pidgeon Hole Principle) the output may be NONE of the vectors that are memorized in the network In other words: among the offered M choices, we could not decide

    18. Main Problems with Hopfield Network What if vectors xm* to be learned are not exact (contain error)? In other words: If we had two patterns representing class 1 and class 2, we could assign each pattern to a vector and learn the vectors However, it we had 100 different patterns representing class 1, and 100 patterns representing class 2, we cannot assign one vector to each pattern

    19. Show Matlab demo for Hopfield Networks

More Related