Member-only story
Understanding the Hopfield Network with Examples
How the eponymous network contributed to AI development
The Hopfield network played an important role in the development of AI. The contribution was awarded with a Nobel Prize in Physics for John Hopfield.
In this article, I want to explain the Hopfield network in simple terms and show how it works with actual examples.
I will conclude by acknowledging its contribution to the modern neural networks that are the current AI boom.
1. Hebbian Learning
Donald Hebb in his 1949 book The Organization of Behavior described a learning theory that laid the foundation of neural networks.
The theory is often summarized as “Neurons that fire together, wire together.” To be precise, the Hebbian learning theory says that the activation of one neuron reinforces the firing of another connected neuron.
The weight change between the two neurons xᵢ and xⱼcan be expressed as:
- Δwᵢⱼ: the change in the weight between input xᵢ and xⱼ.
- η: the learning rate.
- xᵢ: the input from the…
