Okay, good. Okay, so I'm going to talk about something unusual today. I'm going to talk about Hopfield and Boltzmann machines because that's recently become topical. Um, these are things from the history of neural nets that most people who do neural nets now don't know anything about. Um, so I can assume you don't know anything about them, but you do know something about machine learning and neural nets.
Okay, so John Hopfield invented things called Hopfield nets, and Hopfield nets are composed of binary threshold units with recurrent connections between them. Now, in general, when you have a recurrent network, it can behave in all sorts of different ways. It can settle to a stable state, it can oscillate, it can be chaotic, so you can't predict what it'll do. But Hopfield realized that if you make this connection symmetric, then there's a global energy function, and the symmetry of the connections is equivalent to Newton's third law. It's equivalent to saying action and reaction are equal and opposite. That's what you need for a global energy function.
So now each binary configuration of the whole network has an energy, and what I mean by a binary configuration is an assignment of ones and zeros to all the neurons in the network. That's a configuration. And the binary threshold decision rule is going to cause a unit to adopt whatever state minimizes the energy. So if we have a whole bunch of neurons, there's an energy function that's the sum of many different contributions, and this energy function is going to control what happens when the network settles down. So the energy function has a term to do with the biases of the individual neurons, and we'll ignore that for now. And then the main term is to do with interactions between neurons. And so that second term says that there's a weight i j between neurons I and neuron J, and there's the states of those two neurons which are either ones or zeros. And so that second term only comes into play if both of the neurons are on, and if both of the neurons are on, the weight between them contributes to the negative energy, or what I call the goodness. So a good state is one with lots of pairs of units on that have big weights between them, big positive weights between them. Um, and we're ignoring the bias term for now.
So this quadratic energy function makes it possible for each neuron to know how it's contributing to the global, how its state contributes to the global energy. So for neuron I, it'll have what I call an energy gap, which is, given the states of all the other neurons, there'll be the energy of the whole network if you were to turn neuron I off, minus the energy of the whole network if you turn neuron I on. That's the energy gap of the neuron. It's how the energy changes as you change the state of the neuron. And that energy gap is just the bias of neuron I plus the sum over all connections coming from other neurons J of the weight on the connection. And that, of course, is exactly what a binary threshold neuron computes. It adds up the input coming from all the other active neurons, the ones whose state is one, adds in the bias, and then if that's above zero, it turns on, and if it's below zero, it turns off. And that's the energy gap, that difference in the energy depending on whether it's off or on.
So here's a little example of a Hopfield net. Um, we can give it states. So I've now assigned states to all the neurons, and if I want to compute the energy of that state, or the negative energy which I call the goodness, it's just the sum over all pairs of units of the product of their states times the weight between them. So in this net, there's only one pair of units that's both on, and they have a weight of three between them. So that gives us an energy of minus three, or a goodness of three. It's often easier to talk about goodness.
Now we could update the states of the network by picking a neuron in the network, like that one for example, and we could now ask, what's the total input to that neuron? Well, in this network, I haven't put in any biases, so the total input is whatever it gets from other active neurons. So it gets an input of minus four from the active neuron it's connected to. The other neuron it's connected to is off, so its total input is minus four. So it just will stay in the same state. It'll stay at zero, and the goodness is still three. We could pick another neuron, I think, yes, like that one. If we look at that one, its total input is three from the other neurons active, and that's it. So it's got a positive input, so it will stay on. This is getting quite boring. We'd like one that changes. So let's pick that one. That one was off, but if you look at the total input it's getting now, it's getting plus two from one of the neurons that's on, minus one from the other one. So it's got a total input of plus one. So we'll flip it to be on because it's got positive input. And when we flip it to be on, the goodness goes to four, the energy goes to minus four. So we've got a better state of the network. And the state the network's in now, with that triangle of units being on, the kind of left side triangle, that's an energy minimum. If you look around now and ask, can I flip any unit to make things better, you can't. Um, that's a local optimum, a local minimum of the energy.
But if you take this same network, there's actually a deeper energy minimum, which is having the right-hand triangle on. Um, so the net's got two triangles. It's a bit like the American political system in these triangles. Um, you tend to have positive weights between neurons, and the two triangles hate each other, so you have that big negative weight of minus four between the two triangles, even though one unit is involved in both triangles. Um, and so the two minima of this network are with the right-hand triangle on, which has a goodness of five, that's the deepest minimum, and with the left-hand triangle on, which has a goodness of four, not quite so good.
Okay, I just said all that. Um, so that's the global minimum. And Hopfield thought of a way to make use of networks like that. So what Hopfield established is if you make symmetric connections and you use the binary threshold decision rule, and you just go around updating neurons one at a time, the network will settle to an energy minimum, not necessarily the global minimum. He thought you could use this in biology, um, to store memories. You could think of memories as energy minima of a neural net like this. It's obviously a highly idealized neural net, but it's a network of binary neurons that are either on or off, um, with pairwise weights between them. And then what can happen is if you take a memory and you corrupt it, or you leave out part of it, you say some of the neurons, I don't know what their states are, they're kind of neutral, or I just take a memory and flip the states of some units, it can correct the corruptions, or it can fill out incomplete memories.
Now that idea of memories as energy minima was proposed a long time ago by I. A. Richards in a book on literary criticism. Um, Hopfield didn't know about that, so, um, there's no reason why Hopfield should have referred to that book. Um, but Schmid, who of course would have said he was a crook because he didn't, um, energy minima, um, represent memories, and you get a content-addressable memory. So you can access an item in memory just by knowing some parts of it. You know the states of some units, and now this network will fill out the rest of the memory. This was done in the year 16 before Google, and in the year 16 before Google, content-addressable memory was not something we were used to. You couldn't normally access things by just giving a small part of their content and getting the rest. Of course, as soon as Google came along, you could access documents by giving a few rare words in the document, and you get the document. Um, but this was an early example of a content-addressable memory, and it was biologically more sensible. Um, it's robust against hardware damage, and it's a bit like having a few dinosaur bones and reconstructing the whole dinosaur, which people do all the time.
So the way you store memories in a Hopfield net depends on whether the units have states of one and minus one. I'll often say units instead of neurons. Um, states of one and minus one, or states of one and zero. If they have states of one and minus one, the storage rule is very easy. You just change the weight by the product of the activities of those two neurons in the memory. So if they've got opposite states, you reduce the weight, and if they've got the same state, you increase the weight. That seems a sensible thing to do, and that'll make a Hopfield memory that works. If they've got, it's a very simple rule, it doesn't involve iterating at all. You just store memories in one shot. If they've got, um, weights that have states of zero and one, the rule gets a bit more complicated, and there's actually several versions of the rule, but this will be one version. You change the weight by the product of those two terms, and each of those terms is either minus a half or plus a half, depending on whether the neuron is on or off. So if S I is off, you'll get minus half for the first term, and if S J is on, you'll get plus half for the second term, and so you'll decrease the weight between the two neurons. They'll kind of inhibit each other.
Okay, so what people discovered with Hopfield nets, a lot of research was done by physicists. Um, they discovered that, um, one thing that limits how many memories you can store is spurious minima. So each time you memorize a complete configuration of the network, that is an assignment of ones and zeros to all the neurons in the network, you create an energy minimum. At least you hope you create an energy minimum. Um, and so for example, we might create two different energy minima like this by storing two different memories. But if the memories are close together, you might create a spurious minimum that's the sort of average of the two memories, and that's a problem. Um, we've got now a false memory that's the average of these two memories.
So this limits the capacity of the Hopfield net. And actually, that picture is very bad. Um, because the state space of a Hopfield net is the corners of a hypercube. If there's n neurons, there's an n-dimensional hypercube, and the different configurations are the corners of that hypercube. Um, and so showing it as 1D like this, showing the state space as 1D, is misleading. But you get the point. If you have two similar memories, when you store them, you might actually get the average of those two memories being better than either memory.
So to deal with that, Hopfield, Feinstein, and Palmer suggested that what you should do is put the net in a random state, let it settle, and do unlearning. That way you get rid of deep minima because it will probably settle to one of the deep minima, which might be the spurious states. This seems extremely optimistic. They showed that it could be made to work, but they didn't have any theory of why it worked, and they didn't have any theory of how much unlearning you should do.
Based on that idea, which basically Feinstein's idea, um, Crick and Mitchison proposed that unlearning might be a model of what dreams are for. So when you dream, you don't remember your dreams. That is, the dream you had just before you woke up is in your short-term memory, probably in temporary changes to the weights in your neural net. It'll fade very quickly, but you can remember the last bit of the dream you had just before you woke up. But if we wake you up at random times during the night, we can see you dream for several hours each night, and you don't remember them. And why don't you remember them? Because they're very interesting. Um, my dreams are typically much more interesting than what happens to me during the day, and yet I don't remember them. So Crick's theory was it's because the point of dreams is to unlearn the things you tend to believe that weren't supported by data. But they couldn't say how much unlearning you should do.
And Terry Sejnowski and I got to thinking, can we derive, um, unlearning as the way to minimize some sensible cost function, so we can say how much unlearning you should do and what it is you're optimizing by doing unlearning. And that was then combined with a different theory of what you might use Hopfield nets for. So instead of using them just to store memories, we could use them for perception. And in perception, what you have is you have some inputs, like the pixels of an image, and you want to interpret them, that is, get some interpretation of what the image depicts. Um, so to do that with a Hopfield net, you divide the units into two groups. So there'd be visible units, which is where you put the input, so they'd be like the pixels, and if you turn them on, that's a bright pixel, if you turn them off, that's a dark pixel. And then there'd be hidden units that correspond to what you've interpreted as going on in that image. And the energy of the whole configuration would represent the badness of the interpretation. So as the hidden units kind of, as you update them and they settle down to better states, lower energy states, they'll be getting better interpretations of the input image.
So I want to give you a little concrete example of that. It's, it's not a realistic example, but it'll allow you to get the idea. Suppose I show you some 2D lines in an image, and you want to interpret them as 3D edges. So obviously, in a 2D line, you've lost some information about the 3D edge. You've lost information about the depth, at what depth the ends of that edge are. So there's your eyeball in blue, and there's two rays in red, and I'll show you a whole bunch of 3D edges that will cause exactly the same 2D line in the image. So that one and that one, things where the two ends of the 3D edge end on these two rays, will all lead to the same line in the image. So that's the information you've lost. For the, you know that one of these edges is right, but you don't know the depth on where, what, how deep it is on each of those rays. Okay, um, so there's this whole family of edges that could all be interpretations of the same 2D line in the image.
So we could set up a neural network that knows that. Um, we know you can only see one of those edges at a time if objects are opaque because they get in each other's way. So we can have a line drawing, and we could have a whole bunch of neurons that represent particular 2D lines. Um, worry about efficiency later, this is just to get the ideas over. And so in this image, there's two particular lines, and I'm showing you the two neurons that become active if those lines are present. And in general, only a few of these guys will get active in each image because most of the lines in an image aren't there. And then for each of these lines, we can have a bunch of 3D edges, and these, you should compete. So those green arrows are connections with positive weights that say if you see that line, you should try and activate all of these 3D edges. And the red lines are connections with negative weights that say each of these edges will inhibit all the other edges. And so what you want is a little circuit there that if you activate a line, it'll try and activate these edges, and it'll be happy with any one edge being activated, um, but be unhappy if you activate more than one edge. But it doesn't really know which edge to activate.
We can do the same for another line in the image, and then we can say we have some prior beliefs about objects in the world. For example, we believe that if you see two lines like the lines that these arrows come from at the bottom, then probably if they meet in the image, they meet in depth in 3D. So we could put in a connection between the interpretations of these two lines like that. Um, suppose those are two lines that actually join in 3D, then we'll put a positive connection for them between them. They should support each other. And there might actually be two lines that join in 3D at a right angle, and then we put a big positive connection because we like to see right angles. So there's two lines that join at a right angle, we'd really like those two lines to be active together. And I think you can see that if we put in connections like that, that are really about what we expect objects to look like and how we expect lines to be interpreted, and we make a big network like this, the network might have many stable states that correspond, for example, to flipping that cube in depth, um, like the Necker cube, um, but it ought to settle down into one of those stable states where it tries to maximize how many interpretations of lines join in depth, and if they join in the image, and how many right angles there are.
Okay, so you can make a circuit like that, and you can hope that if you put the right weights on the connections, you'll be able to interpret images in terms of 3D, um, edges. And that raises two issues. One is the search issue: how do you avoid getting trapped in poor local minima? Because if you're doing perception, you want the best interpretation. Um, and the second issue is, well, how do you, how do you, um, learn all of those connection strengths? So let's start with the first issue.
If you want to avoid local minima, it actually pays to have some noise in your decision rule. So in a Hopfield net, you use the binary threshold decision rule, and that will always flip a unit into whatever state reduces energy, and that makes it impossible to escape from local minima. So if you're in local minimum A, you can't get to B because there's no noise in the system. But if you had some random noise, you might kind of jitter out of A and end up in B, and you're more likely to go from A to B than you are to go from B to A. So noise is going to help your chances because the energy barrier is higher from B to get to A than vice versa.
Now there's a technique called simulated annealing that says start with a lot of noise and gradually reduce the amount of noise, because with a lot of noise it's easy to cross barriers, and with a little noise you're very unlikely to go from B to A. And the best tradeoff is to start with a lot of noise and gradually reduce it. That's called simulated annealing, and it was invented by Kirkpatrick and others at about the same time as Hopfield nets were invented. Um, Terry and I got very interested in using stochastic binary units in Hopfield nets and using simulated annealing. Um, so let's look at a stochastic unit. It has something called a temperature, which is the amount of noise, and the noise level has the effect of effectively decreasing energy gaps between configurations. And so the equation looks like that. In a Hopfield net, the temperature is zero. So if you put a temperature of zero there, you'll see that if Delta E is positive, we have e to the negative number, and it's negative infinity, and e to the negative infinity is zero. So if Delta E is positive, we'll have one over one, and the probability of turning the unit on will be one. If Delta E is negative, we'll divide by zero, and we'll have minus infinity, and we have e to the minus infinity, which is, did I get it wrong? Yeah, if Delta E is negative, with the minus sign on the Delta, we get a positive, we get e to the plus infinity, so we get one over one plus e to the infinity, which is zero. So the probability of turning it on will be zero. That's the Hopfield decision rule. It's just this stochastic decision rule at a temperature of zero. And obviously, if you know any physics, the question is what happens when you have a temperature that's not zero, and then you get simulated annealing. So that seemed like a big advance for Hopfield nets, they could do simulated annealing.
Um, I'm just reminding you of the equation for the energy gap, which is just the difference between, um, the energy with the unit off and the energy with the unit on. But simulated annealing turned out to be a distraction. It's still useful, but I don't want to go into all the complications of that. So what we're going to do is just stick with a temperature of one now, so that we can get at the main idea behind Boltzmann machines. So we won't mess with the temperature, we'll just set the temperature to one.
Okay, so now I have to explain thermal equilibrium at a temperature of one. And if you're not a physicist or a statistician, thermodynamic equilibrium is a difficult concept. So what most people initially think it means is the system settled down and nothing's changing. That's not what thermal equilibrium is. When a system reaches thermal equilibrium, things are still changing. Um, any one system at thermal equilibrium is flipping between states, but the probability distribution of finding it in a particular state is settled down. That's a hard thing to think about. Um, statisticians call that the stationary distribution, and physicists call it thermal equilibrium. Um, an intuitive way to think about it is to imagine a whole bunch of systems that are identical, um, but you're using a different random number generator when you decide whether to turn things on or off. Um, so you look at the energy gap. If the energy gap's positive, you typically turn things on, and if it's negative, you typically turn things off. Um, but it's a stochastic decision rule. Um, so if the energy gap's close to zero, you might turn things on or off. And we imagine this huge ensemble of identical systems, each with a different random number generator. They all have the same energy function, but they'll all be in different states, and they'll all be flipping between states. And if you start them off in random states, um, they'll go to other random states, but after a while, if you use that decision rule I showed you for stochastic binary neurons, um, the fraction in any one state will be stable. So you could think of the probability of a configuration as the fraction of this huge ensemble that's in that one state. And what will happen at thermal equilibrium is things in that state will flip to other states, but things in other states will flip to that state, and those two, the loss from that state and the gain from that state, will balance out. And so that's called detailed balance. And so what you'll get is the fraction of the configurations, fraction of the systems in a particular configuration, will be stable. It won't change anymore, even though you're updating the units in all the configurations. That's thermal equilibrium.
I'll give you one more attempt to explain thermal equilibrium. Um, so we can start with any distribution we like over all these identical systems, um, and as we go around in each system, letting units flip on or off stochastically, um, gradually what will happen is we'll settle to thermal equilibrium where the fraction is constant. Um, and I'll give you one more analogy for that.
So suppose you've got this huge casino at Las Vegas, and it's full of card dealers. And we need it, we need many more than 52 factorial of them, so it's rather a large number of card dealers, but for a thought experiment, that's fine. Um, and each card dealer starts off with his pack of cards in exactly canonical order. So it's maybe the Ace of Spades and the King of Spades and the Queen of Spades and so on. And so initially, all of the packs of cards are in the canonical configuration, the one where it's the standard order. Then they start shuffling, and they do random shuffles, not fancy shuffles that come back to the same state again, but just random shuffles. And so now, as they start shuffling, you'll get many more different configurations of the different packs of cards. Um, but soon after they started shuffling, you'll still have the property the King of Spades is quite likely to be next to the Queen of Spades. They haven't sort of split them apart yet in the shuffling. So after they've only been shuffling for a very short time, there'll still be many more configurations of card packs where the King of Spades is next to the Queen of Spades. But if they keep on shuffling for a long time, then after a long enough time, all sequences of cards in these packs will be equally likely. That's thermal equilibrium. They keep on shuffling and they keep on shuffling, and so any one pack of cards changes, but if you look at this whole ensemble of packs, the fraction in any one configuration stays constant.
Now, one thing wrong with that is for packs of cards, all configurations are equally likely, um, unless maybe some cards are heavier than others. But for the systems we're talking about, not all configurations are equally likely, and in that thermal equilibrium, different configurations will have different probabilities.
Okay, so now let's talk about how we could model binary data with, uh, what we call a Boltzmann machine. It's a Hopfield net that has visible units and hidden units, and it has this stochastic decision rule that you probabilistically turn units on or off depending on the energy gap. Um, so what we'd like to do is be able to take a set of binary vectors that are the training data, so imagine those as binary images, and we'd like the model to assign high probability, relatively high probability, to the binary vectors in the training set, and a relatively low probability to all other binary vectors. That's the aim of training now. And remember, that's just for the visible vectors, the ones that we, that form the image, and we've got all these extra units that can be used to help achieve that. So it's much more complicated than the Hopfield net where you get to see everything. You've got all these hidden units which can be used to help you model the binary data on the visible units, but the question is how should you use them. And it looks like that's a very complicated issue.
So with the Boltzmann machine, everything depends on the energies of joint configurations of the visible and hidden units. Um, and the energies of the joint configurations relate to the probabilities of those configurations. And you can think of it in two different ways. So you can say, if I have a joint configuration that has a binary vector v on the visible units and a binary vector h on the hidden units, um, that binary configuration will have an energy, which is just the energy of the joint configuration that has v on the visibles and h on the hiddens. That'll be a number, and the probability of finding the system in that configuration at thermal equilibrium is going to be proportional to e to the minus that energy. That's called the Boltzmann distribution. When the probabilities are proportional to e to the minus the energy, there's another way of thinking about the probability, which is we can say, okay, take this Boltzmann machine and start updating the units stochastically. Um, and you're updating all of the units stochastically, the visible ones and the hidden ones. There's no data you're putting in. You're just letting the Boltzmann machine dream, if you like, um, or generate. And after you reach thermal equilibrium, which might take a long time, you look to see whether the whole system, this Boltzmann machine, has the vector v on the visible units and has the vector h on the hidden units. And the probability that you'll see that will turn out to be exactly equal to this probability that you can compute from the energies, because at thermal equilibrium, it will have reached a Boltzmann distribution.
Now you'll notice I express the probability of v and h as proportional to e to the minus the energy, and that's important. It's not equal to e to the minus, it's just proportional, because you have to normalize probabilities so they add to one. Um, but the point is those two definitions agree. What you could compute if you could consider all possible configurations, and what you could measure if you just run it stochastically.
So the energy of a joint configuration is going to be something that looks like this. There's going to be bias terms, there'll be biases for the visible units and hidden units. There's going to be interactions between visible units. There's going to be interactions between, yeah, that's the energy. Those are the bias terms. There's interactions between visible units, and we use I and J like that, so you with a less than sign, so you don't count them twice, and so you don't count things where I equals J. Um, and then there's going to be weights, and there's going to be interactions between visible and hidden units, and interactions between hidden units. And there's going to be a big energy expression like that. And how the system behaves is going to depend on those biases and those weights.
So if we want to use energies to define probabilities, um, we know that P of v given h is going to be proportional to e to the minus the energy of that joint configuration. But what do we normalize by? Well, we have to normalize by the sum over all possible visible vectors u and all possible hidden vectors g of e to the minus the energy of the configuration with u on the visible units and g on the hidden units. That normalization term is what physicists call a partition function, and it's a...
monster to compute you can't actually compute it for big systems it's just much too big and that's why it's good to be able to sample and get the same answer or roughly the same answer so that's called the partition function and to get the probability the actual probability of a joint configuration you have to compute the energy of that joint configuration and then normalize by the partition function.
If you want the energy of a sorry if you want the probability of a visible vector so suppose we trained up a Boltzmann machine and we're now going to just let it generate data so it's a generative model it's generative AI the probability that it will generate a particular vector on its visible units at thermal equilibrium will be the sum over all the possible hidden configurations of the joint energy of v and h divided by the partition function so that's the computation you'd have to do if analytically you wanted to work out from the weights the probability of this thing having vector v on its visible units when it's generating data is running without any input just generating data on the visible units.
And I'm going to go through a worked example because I find I always understand these things much better if I've seen a worked example so let's have a very simple network with two visible units and two hidden units so there's four possible states of the visible units 1 1 1 0 0 1 and 0 0 and there's four possible states of the two hidden units and there's obviously 16 possible joint states which I've shown you in this table.
And so the top four lines are when the two visible units are on and we look at all possible states of the hidden units and for each of those joint configurations we compute a negative energy that is a goodness and so let's take the top line that's 1 1 1 1 so if we turn everything on then the goodness is going to be the sum of all those three weights because they're all between pairs of units that are on so it's going to be plus two minus one plus one so the goodness is going to be two and we could do the same for all those other joint configurations and we'll get the numbers for e to the minus the energy.
Now and then we can add up all those numbers to get 39.7 that's the partition function we divide by 39.7 and we'll get a bunch of numbers that add to one now so there'll be probabilities and if you want to compute the probability in this particular Boltzmann machine if you run it to thermal equilibrium by updating the states of the units many times what's the probability we'll see 1 1 on the two visible units well it's 0.466.
There's four different ways you could see it depending on the four different states of the hidden units two of them have quite high probability two of them have lower probability you add them all up and the probability that this Boltzmann machine if you run it to thermal equilibrium would exhibit the state 1 1 on the visible units is 0.466.
And if you implement this and try it yeah it's roughly right it's 0.466 of the time when you sample it at equilibrium you'll see a 1 1 seeing 0 is going to be much rarer you'll see that about 0.084 of the time but I've given you a detailed example of how you turn the weights in the network into energies of joint configurations you turn those into each of the minus the energies you turn those into probabilities by normalizing you integrate over all the states of the hidden units and now you can get the probabilities of visible units.
So that's how a Boltzmann machine that's how these three numbers the plus two the minus one and the plus one model a probability distribution over visible vectors and obviously if you had a training set that said the vectors 1 1 and 1 0 quite common and the vectors 0 1 and 0 0 quite rare this would be not such a bad model because it would make the two that are quite common common and the two that are quite rare rarer.
Okay to get a sample from the model what you do is you can't do that computation analytically that I just did in a big network but what you can do is just use Markov chain Monte Carlo which amounts to you go around to all these units the visible ones and the hidden ones pick one unit at a time use the stochastic binary decision rule to decide should it be on or off and just keep doing that until you reach the equilibrium and then look at the visible units and you got a sample from the model.
That's just what I said and you do that at a temperature of one so you've got a sample from the model running at a temperature of one and that the probability of getting any particular sample will be the same as the probability you could have computed if you could do the computation which you can't because it's too big so the probability of getting a joint configuration of the visible and hiddens would look like that.
If you want a sample of the hiddens given a particular visible vector and what you do is you clamp the visible vector on the visible units so you don't sample those you just sample the hidden units and now you'll be sampling from interpretations of that input think of the clamped visible vector as like a line drawing and think of the hidden units as what the edge units are doing and you can sample interpretations now of that line drawing.
So it's just like sampling complete configurations which you need to do to sample visible vectors but you're only going to allow the hidden units to flip because it's going to be with a particular visible vector I should say we're almost done.
And you require those samples from the posterior given a visible vector in order to learn the weights now you might think and everybody thought it would be extremely complicated to learn the weights in this network and I'll give you one reason for believing that so our goal in learning is to set weights so that when you sample from this model by updating the hidden units and updating the visible units and keeping doing that until you reach thermal equilibrium and then look at a sample on the visible units when you sample that way you get probabilities you get high probabilities for the training data and low probabilities for everything else.
So if you want to maximize the product of the probability of the training data is equivalent to maximizing the sum of the log probabilities of the training data and we're now going to see why it's possible that you could get a simple learning rule.
So we let the network settle to its stationary distribution and we sample visible vectors and we ask how are the probabilities of those visible vectors related to the energies in the network well I'll come back to that I first wanted to show you why you might think it's very hard to learn the weights.
So suppose I have training data that says this network with two visible units should frequently have 1 0 as the state of the visible units and frequently have 0 1 but only very rarely have 0 0 or 1 1.
If you just use intuitive common sense it's obvious that you'd like that chain of weights that go from the first visible unit to the last visible unit the other visible unit that chain of five weights you'd like the products of all those weights to be negative if the product of all those weights is negative then turning the first unit on will tend to turn the second unit off and vice versa the symmetric weights so we know we want the product to be negative.
So if you now look at one of the weights like W1 and you ask so how should I change that to improve things well you want to make the product negative and so if all the other weights were positive you want to make W1 negative if one of the other weights was negative you want to make W1 positive so how you want to change W1 depends on the value of W3 which is a remote weight it doesn't even involve the same units as W1 connects.
So we know that in order to know how to change W1 you need to know the sign of W3 now in an algorithm like back propagation you send information backwards to get that information back but in the Boltzmann machine you've got a completely different way of getting that information which depends on something that looks just like magic.
So here's the very surprising fact everything one weight needs to know about all the other weights is conveyed in can be obtained locally by looking at two correlations you don't need anything like the back pass of back propagation.
So suppose I got a visible vector v and I want to make that visible vector more likely to be generated by the network so I give it an example of an image and I say how I ask how should I change the weight w_ij between two neurons so as to make v more likely and it might be between two visible neurons or between a visible neuron and hidden neuron or between two hidden neurons it's the same rule.
And the derivative you want that will make the log probability of v higher is just the difference between how often unit i and unit j are on together when you're clamping v on the visible units and how often they're on together when you don't clamp anything on the visible units you just let it dream you run it where you're updating all of the units and you run it into thermal equilibrium and you just look to see how often those two units are on together.
So what you've got here is an incredibly simple learning rule it says to learn the connection strength between neurons i and j you don't need anything like back propagation which is not very neurally plausible because it has to propagate a different kind of information in the back pass from the forward pass you've got something really simple that says just while you're awake and you've got visible vectors coming in just measure the correlation of i and j that's what those angle brackets mean they're the physicist notation for a correlation measure how often i and j are on together then go to sleep and let the model generate data.
So you're updating all the units now not letting some of them be driven by sensory input and measure the same thing measure how often those two units are on together and that difference in correlations is the learning rule so this is just a beautiful learning rule it's very very simple it looks like you can put it into a brain very easily and instead of going forwards and then backwards like you do in back propagation which can't be done in real time you have to go forwards and then you have to stop and go backwards which isn't very useful for perception this algorithm says no no you don't do that you have a phase when you're awake when data is coming in and you're just measuring the correlations between all pairs of units that are connected and then you have a different phase when you're asleep and you're generating data from the model and you're measuring the correlations and the difference of those correlations is the learning signal.
And of course there's that minus sign there and what that minus sign says is when you're asleep if two units are on together you decrease the strength of the connection that is you're making the joint configurations you get when you're asleep be less likely and you're making the configurations you get when you're awake be more likely so this is just like the Crick theory that you're unlearning during sleep but this is actually the derivative of something this learning rule it's the derivative of exactly the thing you want which is the log probability of generating a training vector.
Now you can ask why is it so simple I said all that the reason it's so simple that the learning rule is just proportional to that difference in correlations when you're clamping the visible units or when you're not clamping them with data the reason it's simple is this at thermal equilibrium you have this magical property that the log probability of a configuration is a linear function of its energy.
To put it another way the probability is e to the minus the energy so take logs the log probability is linear in the energy and the energy is a linear function of the weights so if you take the energy which is s_i s_j w_ij and you differentiate it with respect to w_ij you just get s_i s_j and so you're getting the derivative of the energy with respect to a weight is this s_i s_j and because the process to thermal equilibrium propagates information about the weights you get this magical property you don't need a back propagation phase you can just do it by correlations you get when you're awake and correlations you get when you're asleep.
And if you think about why you need a negative phase let's go back to the math the probability of a visible vector I know I'm over time the probability of a visible vector is the sum over all hidden configurations of that visible vector on the visible units and any old hidden configuration h on the hidden units so there's a summation there normalized by the partition function which is a sum over all possible visible vectors and all possible hidden vectors of e to the minus that visible vector that hidden vector.
And if you think about what you should do to maximize the probability of v you should obviously lower the energy raise the goodness of the combination of v with any of the hidden vectors particularly ones that are already good and so what you're doing in the positive phase of the learning algorithm when you're awake is with each vector v you're trying to fill it in with the hidden vector that goes well with it that is a sensible interpretation of v and then you're trying to make that combination more plausible by lowering the energy of that combination.
But there's this normalization term and you're not going to make it more plausible if you make everything more plausible what you have to do is find all the competing combinations these u's and g's and make all of them less likely and that's what the negative term in the learning rule is doing is making the partition function smaller while making the sum of the goodnesses with this particular hidden vector bigger with this particular visible vector bigger.
So the positive phase of the learning while you're awake maximizes the top line and the negative phase minimizes the bottom line and so these energy based models have to have two phases like that because you can't compute this partition function it's much too big you have to sample it and so immediately this kind of energy based model makes sense of why you would want to go to sleep and dream and not remember your dreams.
And that's the end of my talk so I'll try and go back to seeing whether I can see people again thank you very much for the talk.
