The surprising truth about AI’s limits and potential | Geoffrey Hinton | Strange Loop

Have you reflected a lot on how to select talent or has that mostly been intuitive to you? Ilya just shows up and you're like, this is a clever guy, let's, let's work together. Or have you thought a lot about that? Should we roll this? Yeah, let's roll this.

Sound is working. So I remember when I first got to Carnegie Mellon from England, in England at a research unit, it would get to be six o'clock and you'd all go for a drink in the pub. Um, at Carnegie Mellon, I remember after I'd been there a few weeks, it was Saturday night. I didn't have any friends yet and I didn't know what to do. So I decided I'd go into the lab and do some programming 'cause I had a list machine and you couldn't program it from home. So I went into the lab at about nine o'clock on a Saturday night and it was swarming. All the students were there and they were all there because what they were working on was the future. They all believed that what they did next was gonna change the course of computer science and it was just so different from England. And so that was very refreshing.

Take me back to the very beginning, Geoff at Cambridge. Uh, trying to understand the brain. Uh, what was that like? It was very disappointing. So I did physiology and in the summer term they were gonna teach us how the brain worked and it, all they taught us was how neurons conduct action potentials, which is very interesting, but it doesn't tell you how the brain works. So that was extremely disappointing. I switched to philosophy then, I thought maybe they'd tell us how the mind worked and that was very disappointing. I eventually ended up going to Edinburgh to do AI and that was more interesting. At least you could simulate things so you could test out theories.

And did you remember what intrigued you about AI? Was it a paper? Was it any particular person that exposed you to those ideas? I guess it was a book I read by Donald Hebb that influenced me a lot. Um, he was very interested in how you learn the connection strengths in neural nets. I also read a book by John von Neumann early on, um, who was very interested in how the brain computes and how it's different from normal computers. And did you get that conviction that this ideas would work out at that point, or what was your intuition back in the Edinburgh days?

It seemed to me there has to be a way that the brain learns and it's clearly not by having all sorts of things programmed into it and then using logical rules of inference, that just seemed to me crazy from the outset. Um, so we had to figure out how the brain learned to modify connections in a neural net so that it could do complicated things. And von Neumann believed that. Turing believed that. So von Neumann and Turing were both pretty good at logic, but they didn't believe in this logical approach.

And what was your split between studying the ideas from, from neuroscience and just doing what seemed to be good algorithms for, for AI? How much inspiration did you take early on? So I never did that much studying in neuroscience. I was always inspired by what I learned about how the brain works. That there's a bunch of neurons, they perform relatively simple operations, they're nonlinear, um, but they collect inputs, they weight them and then they give an output that depends on that weighted input. And the question is how do you change those weights to make the whole thing do something good? It seems like a fairly simple question.

What collaborations do you remember from from that time? The main collaboration I had at Carnegie Mellon was with someone who wasn't at Carnegie Mellon. I was interacting a lot with Terry Sejnowski who was in Baltimore at Johns Hopkins. And about once a month, either he would drive to Pittsburgh or I would drive to Baltimore. It's 250 miles away and we would spend a weekend together working on Boltzmann machines. That was a wonderful collaboration. We were both convinced it was how the brain worked. That was the most exciting research I've ever done. And a lot of technical results came out that were very interesting, but I think it's not how the brain works. Um, I also had a very good collaboration with um, Peter Brown, who was a very good statistician and he worked on speech recognition at IBM and then he came as a more mature student to Carnegie Mellon just to get a PhD.

Um, but he already knew a lot. He taught me a lot about speech and he in fact taught me about hidden Markov models. I think I learned more from him than he learned from me. That's the kind of student you want. And when he taught me about hidden Markov models, I was doing backdrop with hidden layers and they weren't called hidden layers then. And I decided that name they use in Hidden Markoff models is a great name for variables that you dunno what they're up to. Um, and so that's where the name 'hidden' in neural nets came from me and Peter decided that was a great name for the hidden layers in neural nets. Um, but I learned a lot from Peter about speech.

Take us back to, um, Ilya showed up at your office. I was in my office probably on a Sunday. Um, and I was programming I think, and there was a knock on the door, not just any knock, but it went kind of [knocks on table] sort of an urgent knock. So I went and answered the door and this was this young student there and he said he was cooking fries over the summer, but he'd rather be working in my lab. And so I said, well why don't you make an appointment and we'll talk. And so he just said, "How about now?" And that sort of was Ilya's character. So we talked for a bit and I gave him a paper to read, which was the nature paper on backpropagation. And we made another meeting for a week later and he came back and he said, "I didn't understand it", and I was very disappointed. I thought he seemed like a bright guy, but it's only the chain rule. It's not that hard to understand. And he said, "Oh no, no, I understood that! I just don't understand why you don't give the gradient to a sensible function optimizer", which took us quite a few years to think about. Um, and it kept

on like that with, he had very good, his raw intuitions about things were always very good. What do you think had enabled those, uh, those intuitions for, for Ilya? I don't know. I think he always thought for himself, he was always interested in AI from a young age. Um, he's obviously good at math, so, but it's very hard to know. And what was that collaboration between uh, the two of you like? What part would you play and what part would Ilya play? It was a lot of fun. Um, I remember one occasion when we were trying to do a complicated thing with producing maps of data where I had a kind of mixture model. So you could take the same bunch of similarities and make two maps so that in one map, bank could be close to greed and in another map, bank could be close to river. Um, 'cause in one map you can't have it close to both, right? 'cause river and greed along way part. So we'd have a mixture maps and we were doing it in MATLAB and this involved a lot of reorganization of the code to do the right matrix multiplies.

And Ilya got fed up with that. So he came one day and said, um, "I'm gonna write an interface for MATLAB. So I program in this different language and then I have something that just converts it into MATLAB". And I said, "No Ilya, that'll take you a month to do. We've gotta get on with this project. Don't get diverted by that. And Ilya said, "It's OK, I did it this morning" That's uh, that's quite, quite incredible. And throughout those years, the biggest shift wasn't necessarily just the algorithms, but also the scale. How did you sort of view that scale? Uh, over, over the years?

Ilya got that intuition very early. So Ilya was always preaching that, um, "You just make it bigger and it'll work better". And I always thought that was a bit of a cop-out that you're gonna have to have new ideas too. It turns out Ilya was basically right. New ideas help, things like transformers helped a lot, but it was really the scale of the data and the scale of the computation. And back then we had no idea computers would get like a billion times faster. We thought maybe they'd get a hundred times faster. We were trying to do things by coming up with clever ideas that would've just solved themselves if we'd had bigger scale of the data and computation. In about 2011, Ilya and another graduate student called James Martins and I had a paper using character level prediction. So we took Wikipedia and we tried to predict the next HTML character and that worked remarkably well and we were always amazed at how well it worked. And that was using a fancy optimizer on GPUs and we could never quite believe that

it understood anything, but it looked as though it understood and that just seemed incredible. Can you take us through how are these models trained to predict the next word and why is it the wrong way of, of thinking about them? OK. I don't actually believe it is the wrong way. So in fact, I think I made the first neural net language model that used embeddings and backpropagation. So it's very simple data just triples and it was turning each symbol into an embedding, then having the embeddings interact to predict the embedding of the next symbol and then from that predict the next symbol. And then it was backpropagating through that whole process to learn these triples. And I showed it could generalize. Um, about 10 years later, Yoshua Bengio used a very similar network and showed it worked with real text. And about 10 years after that linguists started believing in embeddings. It was a slow process. The reason I think it's not just predicting the next symbol is if you ask, "Well what does it take to predict the next symbol?"

particularly if you ask me a question and then the first word of the answer is the next symbol. Um, you have to understand the question. So I think by predicting the next symbol, it's very unlike old fashioned autocomplete un old fashioned autocomplete, you'd store sort of triples of words and then if you saw a pair of words, you see how often different words came third. And that way you could predict the next symbol. And that's what most people think autocomplete is like. It's no longer a tool like that, um, to predict the next symbol. You have to understand what's being said. So I think you're forcing it to understand by making it predict the next symbol. And I think it's understanding in much the same way we are. So a lot of people will tell you these things aren't like us. Um, they're just predicting the next symbol. They're not reasoning like us, but actually in order to predict the next symbol, it's gonna have to do some reasoning. And we've seen now that if you make big ones without putting in any special stuff to do reasoning, they can already do some reasoning.

And I think as you make them bigger, they're gonna be able to do more and more reasoning. Do you think I'm doing anything else than predicting the next symbol right now? I think that's how you're learning. I think you're predicting the next video frame. Um, you're predicting the next sound. Um, but I think that's a pretty plausible theory of how the brain's learning. What enables these models to learn such a wide variety of fields. What these big language models are doing is they're looking for common structure, and by finding common structure they can encode things using the common structure and that's more efficient. So let me give you an example. If you ask GPT-4, "Why is a compost heap like an atom bomb?" Most people can't answer that. Most people haven't thought they think atom bomb and compost heaps are very different things. But GPT-4 will tell you, well, the energy scales are very different and the timescales are very different. But the thing that's the same is that when the compost heep gets hotter, it generates heat faster. And when the atom bomb produces more

neutrons, it produces more neutrons faster. And so it gets the idea of a chain reaction. And I believe it's understood they're both forms of chain reaction. It's using that understanding to compress all that information into its weights. And if it's doing that, then it's gonna be doing that for hundreds of things where we haven't seen the analogies yet, but it has and that's where you get creativity from, from seeing these analogies between apparently very different things. And so I think GPT-4 is gonna end up—when it gets bigger—being very creative.

I think this idea that it's just regurgitating what it's learned, just pastiching together text, it's learned already that's completely wrong. It's gonna be even more creative than people I think. You'd argue that it won't just repeat the human knowledge we've developed so far but could also progress beyond that. I think that's something we haven't quite seen yet. We've started seeing some examples of it, but to a large extent we're sort of still at the current level of science. What do you think will enable it to go beyond that?

Well, we've seen that in more limited context. Like if you take AlphaGo in that famous competition with Lee Sedol, um, there was move 37 where AlphaGo made a move that all the experts said must have been a mistake, but actually later they realized it was a brilliant move. Um, so that was created within that limited domain. Um, I think we'll see a lot more of that as these things get bigger. The difference with uh, AlphaGo as well was that it was using reinforcement learning that that subsequently sort of enabled it to, to go beyond the current state. So it started with imitation learning, watching how humans play the game and then it would through self play develop will be beyond that. Do you think that's the missing component of the current data?

I think that may, I think that may well be a missing component, yes. That the self play in AlphaGo and AlphaZero are a large part of why it could make these creative moves. But I don't think it's entirely necessary. So there's a little experiment I did a long time ago where you, you're training a neural net to recognize handwritten digits. I love that example, the MNIST example. And you give it training data where half the answers are wrong. Um, and the question is how well will it learn? And you make half the answers wrong once and keep them like that. So it can't average away the wrongness by just seeing the same example. But with the right answer sometimes and the wrong answer, sometimes when it sees that example half of the examples, when it sees the example, the answer is always wrong.

And so the training data has 50% error, but if you train up backpropagation, it gets down to 5% error or less. In other words, from badly labeled data, it can get much better results. It can see that the training data is wrong and that's how smart students can be smarter than their advisor. And their advisor tells 'em all this stuff and for half of what their advisor tells 'em, they think no rubbish, and they listen to the other half and then they end up smarter than the advisor. So these big neural nets can actually do, they can do much better than their training data and most people don't realize that.

So how, how do you expect these models to add reasoning into them? So I mean one approach is you add sort of the heuristics on on top of them, which a lot of the research is doing now where you have sort of train of thought, you just feedback it's reasoning, um, in into itself. And another way would be in the model itself, uh, as you scale it up. Uh, what's your intuition around that? So my intuition is that as we scale up these models that get better at reasoning and if you ask how people work roughly speaking, we have these intuitions and we can do reasoning and we use the reasoning to correct our intuitions. Of course we use the intuitions during the reasoning to do the reasoning, but it's the conclusion of the reasoning conflicts with our intuitions. We realize the intuitions need to be changed. That's much like in AlphaGo or AlphaZero where you have an evaluation function, um, that just looks at a board and says, how good is that for me?

But then you do the Monte Carlo rollout and now you get a more accurate idea and you can revise your evaluation function. So you can train it by getting it to agree with the results of reasoning. And I think these large language models have to start doing that. They have to start training their raw intuitions about what should come next by doing reasoning and realizing that's not right. And so that way they can get more training data than just mimicking what people did. And that's exactly why AlphaGo could do this creative move 37, it had much more training data 'cause it was using reasoning to check out what the right next move should have been.

And what do you think about multimodality? So we spoke about these analogies and often the analogies are way beyond what we could see. It's discovering analogies that are far beyond the humans and at maybe abstraction levels that we will never be able to understand. Now when we introduce images to that and video and sound, how do you think that will change the models and uh, how do you think it'll change the analogies that it will be able to make? Um, I think it'll change it a lot. I think it'll make it much better at understanding spatial things. For example, from language alone, it's quite hard to understand some spatial things, although remarkably GPT-4 can do that even before it was multimodal. Um, but when you make it multimodal, if you have it both doing vision and reaching out and grabbing things, it'll understand object much better if you can pick them up and turn them over and so on. So although you can learn an awful lot from language, it's easier to learn if you are multimodal and in fact you then need less language and there's

an awful lot of YouTube video for predicting the next frame, so, or something like that. So I think these multimodal models are clearly gonna take over. Um, you can get more data that way. They need less language. So there's really a philosophical point that you could learn a very good model from language alone, but it's much easier to learn it from a multimodal system. And how do you think it'll impact the model's reasoning? I think it'll make it much better at reasoning about space. For example, reasoning about what happens if you pick objects up, if you actually try picking objects up, you're gonna get all sorts of training data that's gonna help.

Do you think the human brain evolved to work well with with language or do you think language evolved to work well with the human brain? I think the question of whether language evolved to work with the brain or the brain evolved to work with language, I think that's a very good question. I think both happened, I used to think we would do a lot of cognition without needing language at all. Um, now I've changed my mind a bit. So let me give you three different views of language, um, and how it relates to cognition. There's the old fashioned symbolic view, which is cognition consists of having strings of symbols in some kind of cleaned up logical language where there's no ambiguity and applying rules of inference. And that's what cognition is. It's just these symbolic manipulations on things that are like strings of language symbols. Um, so that's one extreme view.

An opposite extreme view is no, no, once you get inside the head it's all vectors. So symbols come in, you convert those symbols into big vectors and all the stuff inside is done with big vectors. And then if you want to produce output, you produce symbols again. So there was a point in machine translation in about 2014 when people were using neural recurrent neural nets and words will keep coming in and they'd have a hidden state and they keep accumulating information in this hidden state. So when they got to the end of a sentence that have a big hidden vector that captured the meaning of that sentence, that could then be used for producing the sentence in another language—that was called a thought vector. And that's the sort of second view of language.

You convert the language into a big vector that's nothing like language and that's what cognition's all about. But then there's a third view, which what I believe now, which is that you take these symbols and you convert the symbols into embeddings and you use multiple layers of that. So you get these very rich embeddings, but the embeddings are still tied to the symbols in the sense that you've got a big vector for this symbol and a big vector for that symbol. And these vectors interact to produce the vector for the symbol for the next word. And that's what understanding is. Understanding is knowing how to convert the symbols into these vectors and knowing how the elements of the vector should interact to predict the vector for the next symbol. That's what understanding is, both in these big language models and in our brains. And that's an example which is sort of in between. You're staying with the symbols, but you are interpreting them as these big vectors. And that's where all the work is and all the knowledge

is in what vectors you use and how the elements of those vectors interact not in symbolic rules. Um, but it's not saying that you get away from the symbols altogether. It's saying you turn the symbols into big vectors, but you stay with that surface structure of the symbols. And that's how these models are working. And that now seems to me a more plausible model of human thought too. You were one of the first folks to get the idea of using GPUs and uh, I know Jensen loves you, uh, for that. Uh, back in 2009 you mentioned that you told Jensen that this could be quite a good idea, um, for training neural nets. Take us back to that early intuition of using GPUs for training neural nets.

So actually I think in about 2006 I had a former graduate student called Rick Zelisky, who's a very good computer vision guy. And I talked to him at a meeting and he said, you know, you ought to think about using graphics processing cards because they're very good at matrix multiplies and what you're doing is basically all matrix multiplies. So I thought about that for a bit. And then we learned about these Tesla systems that had, um, four GPUs in and initially we just got, um, gaming GPUs and discovered they made things go 30 times faster. And then we bought one of these Tesla systems with four GPUs and we did speech on that and it worked very well. And then in 2009 I gave a talk at NIPS and I told a thousand machine learning researchers, "You should all go and buy Nvidia GPUs. They're the future.

You need them for doing machine learning". And I actually, um, then sent mail to Nvidia saying, "I told a thousand machine learning researchers to buy your boards, could you give me a free one?" And they said, "No". Actually, they didn't say no, they just didn't reply. Um, but when I told Jensen this story later on, he gave me a free one. That's, uh, that's very, very good. I I think what's interesting is, um, as well is sort of how GPUs has evolved alongside, uh, the the field. So where, where do you think we we should go, uh, go next in the compute?

So my last couple of years at Google, I was thinking about ways of trying to make analog computation so that instead of using like a megawatt, we could use like 30 watts like the brain and we could run these big language models in analog hardware. And I never made it work and, but I started really appreciating digital computation. So if you're gonna use that low power analog computation, every piece of hardware is gonna be a bit different. And the idea is the learning is gonna make use of the specific properties of that hardware. And that's what happens with people. All our brains are different. Um, so we can't then take the weights in your brain and put them in my brain. The hardware's different, the precise properties of the individual neurons are different. The learning used to make—has learned to make use of all that.

And so we are mortal in the sense that the weights in my brain are no good for any other brain. When I die, those weights are useless. Um, we can get information from one to another rather inefficiently by I produce sentences and you figure out how to change your weights. So you would've said the same thing. That's called distillation. But that's a very inefficient way of communicating knowledge. And with digital systems, they're immortal because once you've got some weights, you can throw away the computer, just store the weights on a tape somewhere and now build another computer, put those same weights in and if it's digital it can compute exactly the same thing as the other system did. So digital systems can share weights and that's incredibly much more efficient if you've got a whole bunch of digital systems and they each go and do a tiny bit of learning and they start with the same weights, they do a tiny bit of learning, and then they share their weights again. Um, they all know what all the others learned,

we can't do that. And so they're far superior to us in being able to share knowledge. A lot of the ideas that have been deployed in the field are very old school ideas. It's the ideas that have been around in neuroscience for forever. What do you think is sort of left to apply to the systems that we develop? So one big thing that we still have to catch up with neuroscience on is the timescales for changes. So in nearly all the neural nets, there's a fast timescale for changing activities. So input comes in the activities, the embedding vectors all change, and then there's a slow timescale which is changing the weights and that's long-term learning. And you just have those two timescales.

In the brain, there's many timescales at which weights change. So for example, if I say an unexpected word like "cucumber" and now five minutes later you put headphones on, there's a lot of noise and there's very faint words, you'll be much better at recognizing the word "cucumber" because I said it five minutes ago. So where is that knowledge in the brain? And that knowledge is obviously in temporary changes to synapses. It's not neurons that going "cucumber, cucumber, cucumber". You don't have enough neurons for that.

It's in temporary changes to the weights. And you can do a lot of things with temporary weight changes—fast, what I call fast weights. We don't do that in these neural models. And the reason we don't do it is because if you have temporary changes to the weights that depend on the input data, then you can't process a whole bunch of different cases at the same time. At present we take a whole bunch of different strings, we stack them, stack them together and we process them all in parallel because then we can do matrix, matrix multiplies, which is much more efficient. And just that efficiency is stopping us using fast weights. But the brain clearly uses fast weights for temporary memory and there's all sorts of things you can do that way that we don't do it present. I think that's one of the biggest things we have to learn. I was very hopeful that things like Graphcore, um, if they went sequential and did just online learning, then they could use fast weights. Um, but that hasn't worked out yet. I think it'll work out eventually when people are using conductances for weights.

How has knowing how this models work and knowing how the brain works impacted the way you think? I think there's been one big impact, which is at a fairly abstract level, which is that for many years people were very scornful about the idea of having a big random neural net and just giving it a lot of training data and it would learn to do complicated things. If you talk to statisticians or linguists or most people in AI, they say that's just a pipe dream. There's no way you're gonna learn two really complicated things without some kind of innate knowledge without a lot of architectural restrictions. It turns out that's completely wrong. You can take a big random neural network and you can learn a whole bunch of stuff just from data. Um, so the idea that stochastic gradient descent to adjust the—repeatedly adjust the weights using a gradient that will learn things and will learn big complicated things that's being validated by these big models. And that's a very important thing to know about the brain.

It doesn't have to have all this innate structure. Now obviously it's got a lot of innate structure, but it certainly doesn't need innate structure for things that are easily learned. And so the sort of idea coming from Chomsky that you won't, you won't learn anything complicated like language unless it's all kind of wired in already and just matures. That idea is now clearly nonsense. I'm sure Chomsky would appreciate you calling his ideas, uh, nonsense [laughs]. Well I think actually I think a lot of Chomsky's political ideas are very sensible and I'm always struck by how come someone with such sensible ideas about the Middle East could be so wrong about linguistics?

What do you think would make these models simulate consciousness of humans more effectively? But imagine you had the AI assistant that you've spoken to in your entire life and instead of that being, you know, like ChatGPT today, that sort of deletes the memory of the conversation and you start fresh all of the time. It had self-reflection at some point you, you pass away and you tell that to, to the assistant. Do you think— I mean not me, somebody else tells that to the assistant. Yeah [laughs], it would be difficult for you to tell that to the assistant. Uh, do you think that that assistant would would feel at that point?

Yes. I think they can have feelings too. So I think just as we have this inner theater model for perception, we have an inner theater model for feelings. There are things that I can experience but other people can't. Um, I think that model is equally wrong. So I think suppose I say "I feel like punching Gary on the nose", which I often do. Let's try and abstract that away from the idea of an inner theater. What I'm really saying to you is, um, "If it weren't for the inhibition coming from my frontal lobes, I will perform an action". So when we talk about feelings, we are really talking about um, actions we will perform if it weren't for um, constraints. And that really, uh, that's really what feelings are the actions we would do if it weren't for constraints. Um, so I think you can give the same kind of explanation for feelings and there's no reason why these things can't have feelings.

In fact, in 1973 I saw a robot have an emotion. So in Edinburgh they had a robot with two grippers like this that could assemble a toy car if you put the pieces separately on a piece of green felt. Um, but if you put them in a pile, it's vision wasn't good enough to figure out what was going on. So it put its grippers together and went "whack", and it knocked them so they were scattered and then he could put them together. If you saw that in a person, you say it was cross with a situation 'cause it didn't understand it, so it destroyed it.

That's profound. You uh, when we spoke previously, you described sort of humans and the LLMs as analogy machines. What do you think has been the most powerful analogies that you've found throughout your life? Oh, in—throughout my life? Um, whew! I guess probably a sort of weak analogy that's influenced me a lot is um, the analogy between religious belief and between belief and symbol processing. So when I was very young I was confronted—I came from an atheist family—and went to school and was confronted with religious belief and it just seemed nonsense to me. It still seems nonsense to me. Um, and when I saw symbol processing as an explanation of how people worked, um, I thought it was just the same— nonsense. I don't think it's quite so much nonsense now because I think actually we do do symbol processing. It's just we do it by giving these big embedding vectors to the symbols. But we are actually symbol processing, um, but not at all in the way people thought where you match symbols and the only thing a symbol has is it's identical to another symbol or it's

not identical. That's the only property a symbol has. We don't do that at all. We use the context to give embedding vectors to symbols and then use the interactions between the components of these embedded vectors to do thinking. But there's a very good researcher at Google called Fernando Pereira who said, "Yes, we do have symbolic reasoning and the only symbolic we have is natural language". Natural language is a symbolic language and we reason with it. And I believe that now. You've done some of the most meaningful, uh, research in the history of of computer science. Can you walk us through like how do you select the right problems to, to work on?

Well first let me correct you, me and my students have done a lot of the most meaningful things and it's mainly been a very good collaboration with students and my ability to select very good students. And that came from the fact there were very few people doing neural nets in the seventies and eighties and nineties and two thousands. And so the few people doing neural nets got to pick the very best students. So that was a piece of luck. But my way of selecting problems is basically, well you know, when scientists talk about how they work, they have theories about how they work, which probably don't have much to do with the truth, but my theory is that I look for something where everybody's agreed about something and it feels wrong, just there's a slight intuition that there's something wrong about it. And then I work on that and see if I can elaborate why it is I think it's wrong, and maybe I can make a little demo with a small computer program that shows that it doesn't work the way you might expect.

So let me take one example. Um, most people think that if you add noise to a neural net, it's gonna work worse. Um, if for example, each time you put a training example through, you make half of the neurons be silent, it'll work worse. Actually we know it'll generalize better if you do that and you can demonstrate that. Um, in a simple example, that's what's nice about computer simulations. You can show, you know, this idea you had that adding noise is gonna make it worse and sort of dropping out half the neurons will make it work worse, which you will in the short term. But if you train it like that in the end it'll work better. You can demonstrate that with a small computer program and then you can think hard about why that is and how it stops big elaborate co-adaptations. Um, but that I think that's my method of working. Find something that sounds suspicious and work on it and see if you can give a simple demonstration of why it's wrong.

What sounds suspicious to you now? Well, that we don't use fast weights sound suspicious, that we only have these two timescales. That's just wrong. That's not at all like the brain. Um, and in the long run I think we're gonna have to have many more timescales. So that's an example there. And if you had, if you had your group of, of students today and they came to you and they said, so the Hamming question that we talked about previously, you know: "What's the most important problem in your field?" What would you suggest that they take on and work on next? We spoke about reasoning, timescales. What would be sort of the highest priority problem that that you'd give them?

For me right now it's the same question I've had for the last like 30 years or so, which is "Does the brain do backpropagation?" I believe the brain is getting gradients. If you don't get gradients, your learning is just much worse than if you do get gradients. But how is the brain getting gradients and is it somehow implementing some approximate version of backpropagation or is it some completely different technique? That's a big open question and if I kept on doing research, that's what I would be doing research on.

And when you look back at, at your career now, you've been right about so many things, but what were you wrong about that you wish you sort of spent less time pursuing a certain direction? OK, those are two separate questions. One is "What were you wrong about?" and two, "Do you wish you'd spent less time on it?" I think I was wrong about Boltzmann machines and I'm glad I spent a long time on it. They're a much more beautiful theory of how you get gradients than backpropagation. Backpropagation is just ordinary and sensible and it's just a chamber. Boltzmann machines is clever and it's a very interesting way to get gradients and I would love for that to be how the brain works, but I think it isn't.

Did you spend much time imagining what would happen post these systems developing as well? Did you ever have an idea that, OK, if we could make these systems work really well, we could, you know, democratize education, we could make knowledge way more accessible, um, we could solve some tough problems in medicine. Or was it more to you about understanding the brain? Yes, I, I sort of feel scientists ought to be doing things that are gonna help society, but actually that's not how you do your best research. You do your best research when it's driven by curiosity. You just have to understand something. Um, much more recently I've realized these things could do a lot of harm as well as a lot of good and I've become much more concerned about the effects they're gonna have on society. But that's not what was motivating me. I just wanted to understand how on earth can the brain learn to do things? That's what I want to know. And I sort of failed as a side effect of that failure. We got some nice engineering but...

Yeah, it's, it was a good failure for the world. If you take the lens of the things that could go really right, what do you think are the most promising applications? I think healthcare is clearly a, a big one. Um, with healthcare there's almost no end to how much healthcare society can absorb. If you take someone old, they could use five doctors full time. Um, so when AI gets better than people are doing things, um, you'd like it to get better in areas where you could do with a lot more of that stuff. And we could do with a lot more doctors, if everybody had three doctors of their own, that would be great and we are gonna get to that point.

Um, so that's one reason why healthcare's good. There's also just in new engineering, developing new materials for example, for better solar panels or for superconductivity or for just understanding how the body works. Um, there's gonna be huge impacts there. Those are all gonna be good things. What I worry about is bad actors using them for bad things. We've facilitated people like Putin, or Xi, or Trump using AI for killer robots or for manipulating public opinion, or for mass surveillance. And those are all very worrying things.

Are you ever concerned that slowing down the field could also slow down the positives? Oh absolutely, and I think there's not much chance that the field will slow down, partly because it's international and if one country slows down, the other countries aren't gonna slow down. So there's a race clearly between China and the US and neither is gonna slow down. So yeah, I don't— I mean there was this petition saying we should slow down for six months. I didn't sign it just 'cause I thought it was never gonna happen. I maybe should have signed it 'cause even though it was never gonna happen, it made a political point. It's often good to ask for things, you know, you can't get just to make a point. Um, but I didn't think we're gonna slow down.

And how do you think that it will impact the AI research process, uh, having uh, these assistants? I think it'll make it a lot more efficient. AI research will get a lot more efficient when you've got these assistants that help you program, um, but also help you think through things and probably help you a lot with equations too. Have you reflected much on the process of selecting talent? Has that been mostly intuitive to you? Like when Ilya shows up at the door, you feel this is a smart guy, let's work together.

So, for selecting talent, um, sometimes you just know. So after talking to Ilya for not very long, he seemed very smart and then talking to a bit more, he clearly was very smart and had very good intuitions as well as being good at math. So that was a no-brainer. There's another case where I was at a NIPS conference. Um, we had a poster and I, someone came up and he started asking questions about the poster and every question he asked was a sort of deep insight into what we'd done wrong. Um, and after five minutes I offered him a postdoc position. That guy was David MacKay who was just brilliant and it's very sad he died, but he was, it was very obvious you'd want him. Um, other times it's not so obvious and one thing I did learn was that people are different.

There's not just one type of good student. Um, so there's some students who aren't that creative but are technically extremely strong and will make anything work. There's other students who aren't technically strong but are very creative. Of course you want the ones who are both, but you don't always get that. But I think actually in the lab you need a variety of different kinds of graduate student. But I still go with my gut intuition that sometimes you talk to somebody and they're just very, very, they just get it and those are the ones you want.

What do you think is the reason for some folks having better intuition? Do they just have better training data than than others or how can you develop your intuition? I think it's partly they don't stand for nonsense. So here's a way to get bad intuitions: believe everything you're told. That's fatal. You have to be able to—I think here's what some people do. They have a whole framework for understanding reality and when someone tells 'em something, they try and sort of figure out how that fits into their framework and if it doesn't, they just reject it. And that's a very good strategy. Um, people who try and incorporate whatever they're told end up with a framework that's sort of very fuzzy and sort of can believe everything and that's useless. So I think actually having a strong view of the world and trying to manipulate incoming facts to fit in with your view, obviously it can lead you into deep religious belief and fatal flaws and so on. Like my belief in Boltzmann machines. Um, but

I think that's the way to go. If you've got good intuitions you can trust, you should trust them. If you've got bad intuitions, it doesn't matter what you do, so you might as well trust them. That's a very, very good, uh, very good point. When, when you look at the, the types of research that's, that's that's being done today, do you think we're putting all of our eggs in one basket and we should diversify our ideas a bit more in, in the field? Or do you think this is the most promising direction? So let's go all in on it?

I think having big models and training them on multimodal data, even if it's only to predict the next word, is such a promising approach that we should go pretty much all in it. Obviously there's lots and lots of people doing it now and there's lots of people doing apparently crazy things and that's good. Um, but I think it's fine for like most of the people to be following this path 'cause it's working very well. Do you think that the learning algorithms matter that much or is it just a skill? Are there basically millions of ways that we could, we could get to human level in, in intelligence or are there sort of a select few that we need to discover?

Yeah, so this issue of whether particular learning algorithms are very important or whether there's a great variety of learning algorithms that'll do the job. I don't know the answer. It seems to me though that backpropagation, there's a sense in which it's the correct thing to do. Getting the gradient so that you change a parameter to make it work better. That seems like the right thing to do and it's been amazingly successful. There may well be other learning algorithms that are alternative ways of getting that same gradient or that are getting the gradient to something else and that also work. Um, I think that's all open and a very interesting issue now about whether there's other things you can try and maximize that will give you good systems and maybe the brain's doing that 'cause it's easier, but backprop is in a sense, the right thing to do and we know that doing it works really well.

And one last question. When, when you look back at your sort of decades of research, what are you, what are you most proud of? Is it the students? Is it the research? What, what makes you most proud of when you look back at, at your life's work? The learning algorithm for Boltzmann machines. So the learning algorithm Boltzmann machines is beautifully elegant. It's maybe hopeless in practice. Um, but it's the thing I enjoyed most developing that with Terry and it's what I'm proudest of. Um, even if it's wrong.

What questions do you spend most of your time thinking about now? Is it the— Um, "What should I watch on Netflix?"