视频库 / REC_012
--:--
点击播放，这里会跟随视频显示当前句的中英字幕。
第 12 期 · 回应 Ⅱ·05「会说话，就等于理解吗？」
Do LLMs Understand? AI Pioneer Yann LeCun Spars with DeepMind’s Adam Brown.
Pioneer Works章节 · 点击跳转视频
0:01
杨立昆离职传闻与开场调侃
0:56
神经网络如飞机仿生：原理相似而非复制
3:52
从八十年代神经网络到深度学习更名
6:32
物理学家为何转向研究涌现智能
8:29
快问快答：AI是否智能、是否有意识
11:11
大语言模型如何靠预测下一个词训练
13:19
文本训练量与四岁孩子感官数据之比
18:26
样本效率之争：猫、棋手与奥数金牌
24:51
机器是否真的理解？可解释性证据
28:53
机器学习很烂：像素预测失败与JEPA构想
32:57
LLM是否已饱和：符号推理易，物理直觉难
36:01
五年能力狂飙与递归自我改进的可能
38:48
AI三代人的失败预言：从感知机到专家系统
46:16
简单规则大规模计算带来的涌现复杂性
49:06
目标驱动架构与写入护栏的可控智能
54:15
开源AI与信息垄断：印刷术的类比
57:17
智能体失准、Claude实验与AI安全工程观
63:46
查尔莫斯提问：机器意识的条件与时间表
70:53
新文艺复兴愿景与下一场AI革命
01杨立昆离职传闻与开场调侃
0:01
[music] [music]
[音乐][音乐]
0:13
[music] >> It's a pleasure to have Adam, my colleague and friend, and Jan, who's been with us before. Jan, you really are all over the news right now. Um I've gotten so many people forwarding articles about you this week. It all kicked off on Wednesday. Do you want to discuss the I can just say the headline. The headline was the equivalent of Jan LeCun, chief scientist, leaves Meta. Um do you care to comment? I can neither confirm nor deny. Okay. [laughter] So all of the press uh the the core that's here to get the scoop cannot get the scoop tonight.
[音乐] >> 很高兴请到 Adam，我的同事兼朋友，还有之前也来过的 Jan。Jan，你现在真是到处都是你的新闻。呃，这周有好多人给我转发关于你的文章。这一切是从周三开始的。你想不想聊聊那个——我直接把标题说出来好了。那个标题大意就是：Meta 首席科学家 Yann LeCun 离职。呃，你想说点什么吗？我既不确认也不否认。好吧。[笑声] 所以在场所有想来挖独家的媒体，今晚是挖不到料了。>> [笑声] >> 好吧。呃，你们可以散场后过来请 Jan 喝一杯，看看能套出多少话——我已经喝过一杯了，不过那是
02神经网络如飞机仿生：原理相似而非复制
0:56
>> [laughter] >> All right. Well, um you can come afterwards and buy Jan a drink and see how far you get with um I already had one, but that was a >> [laughter] >> The Frenchman had some wine upstairs. So we have this era where every time any of us turn on the news, look at the computer, read the paper, we're confronted with conversations about the societal implications of AI and whether it's about economic upheaval or um the potential for political manipulation or AI psychosis. There's a lot of pundits out there are discussing this and I and and I it is a very important issue. I kind of want to push that towards the end of our conversation because what a lot of people who are discussing this don't have is the technical expertise that's on this stage. And so I really want to begin by grounding this in that technical scientific conversation. And so I want to begin with you, Jan, about neural nets. Here's this instance of kind of biomimicry where you have these computational neural networks that are
>> [笑声] >> 这位法国人在楼上喝了点红酒。所以我们身处这样一个时代：我们每次打开新闻、看电脑、读报纸，扑面而来的都是关于 AI社会影响的讨论——无论是经济动荡，还是政治操纵的可能性，又或者是所谓的「AI 精神病」。有很多评论人士都在讨论这些，我也觉得这确实是个非常重要的议题。我其实想把这部分留到我们对话的后半段，因为很多参与讨论的人并不具备这个台上所拥有的技术专业性。所以我真的想先从技术和科学层面把这个话题落到实处。那我想先从你开始，Jan，聊聊神经网络。这算是一种仿生学的例子吧——你有这些在模拟人类网络的计算神经网络。你能不能
2:00
emulating human networks. Can you describe to us what that means that a machine is emulating human neural networks? Well, it's not really mimicry. It's more inspiration. The same way, I don't know, airplanes are inspired by by birds, right? The underlying >> work, [clears throat] I thought. Say again? >> But I thought that didn't work. Copying birds with airplanes. Well, in the sense that, you know, airplanes have wings like birds and they generate lift by propelling themselves through the air, but then the analogy stops stops there.
给我们讲讲，机器模拟人类神经网络到底是什么意思？其实这算不上是模仿。更像是一种启发。就好比，我不知道，飞机是受鸟类启发的，对吧？其底层的 >> 原理，[清嗓子] 我原以为。你再说一遍？>> 可我以为那条路走不通啊。照着鸟去造飞机。嗯，从这个意义上说，你知道，飞机像鸟一样有翅膀，靠在空气中推进自己产生升力，但类比也就到此为止了。
2:35
And the wing of an airplane is much simpler than the wing of a bird, but yet the underlying principle is the same. So neural networks are a bit like like that, like are like, you know, our two real brains as airplanes are to birds. They're much simplified in many ways. Um but perhaps some of the underlying principles are the same. We don't actually know because we don't really know the sort of underlying algorithm of the cortex, if you want, or the the the method by which the brain organizes itself and and learns. So we invented substitutes.
飞机的机翼比鸟的翅膀简单得多，但底层原理是一样的。所以神经网络有点像那样，就像，你知道，我们真实的大脑之于神经网络，就如同鸟之于飞机。它们在很多方面都被大大简化了。呃，但也许某些底层原理是相同的。我们其实并不知道，因为我们并不真正了解大脑皮层的底层算法，如果你要这么说的话，或者说大脑自我组织和学习的机制。所以我们发明了替代方案。
3:14
Um sort of like, you know, birds flap their wings and not airplanes, right? They have propellers, so or or turbojets. You know, in in neural nets we have uh learning algorithms and they they allow artificial neural nets to learn in a way that we think is similar to how the brains learn. So the brain is a network of neurons. The neurons are interconnected with each other and the way the brain learns is by modifying the efficacy of the connections between the neurons. And the way a neural net is trained is by modifying the efficacy of the connections between those simulated neurons.
嗯，有点像，你知道的，鸟是靠扇动翅膀飞的，而飞机不是，对吧？飞机有螺旋桨，或者说涡轮喷气发动机。你知道，在神经网络里我们有学习算法，它们让人工神经网络能够以一种我们认为类似于大脑学习的方式来学习。大脑是一个神经元的网络。神经元彼此互相连接，而大脑学习的方式就是改变神经元之间连接的强度。而训练神经网络的方式，就是改变那些模拟神经元之间连接的强度。
03从八十年代神经网络到深度学习更名
3:52
Um each of those is like a we call it a parameter. You you you see this in the press, the number of parameters of a neural net, right? So the the biggest neural net at the moment have you know, hundreds of billions of parameters, uh if not more, and um those are the individual coefficients that are modified by by by training. So And how is deep learning uh emerge in this discussion? Cuz deep learning came along the path after thinking about neural nets. And this has been since the '80s or earlier even. Um yeah, no, '80s roughly. Um So early neural nets uh the the first ones that were capable of of learning or learning something useful at least, you know, in the '50s uh were shallow. You could you could basically train a a single layer of neurons, right? So you would feed the input and train the system to produce a particular output and and you could use those things to kind of recognize or or classify relatively simple patterns, uh but not really sort of complex things. And people at the
嗯，每一个这样的连接就是我们所说的一个参数。你在媒体上会看到，某个神经网络有多少参数，对吧？目前最大的神经网络有，你知道的，几千亿个参数，甚至更多，而这些就是通过训练被不断调整的一个个系数。那么深度学习是怎么进入这个讨论的呢？因为深度学习是在人们思考神经网络之后才出现的。而这从八十年代甚至更早就开始了。嗯，是的，大致是八十年代。最早的神经网络，也就是第一批能够学习、至少能学到点有用东西的网络，是在五十年代出现的，它们都是浅层的。你基本上只能训练单层的神经元，对吧？你把输入喂进去，训练系统输出某个特定的结果，你可以用这类东西来识别或者分类相对简单的模式，但没法处理真正复杂的东西。而当时的人，
4:57
time, even in the '60s, realized that the way to make progress was going to be able to train neural nets with multiple layers. They built neural nets with multiple layers, but they couldn't train all the layers. It would only train the last layer, for example. Um and they didn't really find uh until the 1980s, nobody found really a good way to train those those multi-layer systems uh mostly because the neurons that that they had at the time were the wrong type. Um they had neurons that were binary. So neurons in the brain are binary. They they either fire or they don't fire.
甚至在六十年代就意识到，要取得进展就得能够训练多层的神经网络。他们确实造出了多层的神经网络，但没法训练所有的层。比如说只能训练最后一层。嗯，他们一直没能找到方法，直到八十年代才有人真正找到训练这种多层系统的好办法，这主要是因为他们当时用的神经元类型不对。他们用的是二值神经元。大脑里的神经元是二值的，要么放电，要么不放电。
5:33
Um and people wanted to reproduce that. So they they built simulated neurons that would either be active or inactive. And it turns out for the modern learning algorithms to work, we call them back we call it back propagation, you need to have neurons that have sort of graded responses. Um and uh that only became practical, possible, or people realized it could work in the 1980s. People had the idea before, but they never could really make it work. And so that caused um a renewal of interest in neural nets in the 1980s. They had been largely abandoned in the late '60s and then they came to the fore again in the mid to late '80s. That's when I started kind of my graduate school basically in 1983 and uh there was a wave of interest that lasted about 10 years and then interest went again uh in the mid-'90s until the late 2000 when we rebranded it into deep learning.
嗯，人们想复现这一点。所以他们造出的模拟神经元要么是激活的，要么是不激活的。结果发现，要让现代的学习算法起作用——我们把它叫做反向传播——你需要的神经元得有某种连续的、分级的响应。嗯，这一点直到八十年代才变得可行、可能，或者说人们才意识到它能行得通。之前也有人有过这个想法，但一直没能真正让它跑起来。所以这就带来了八十年代神经网络研究兴趣的复兴。它在六十年代末基本上被抛弃了，然后在八十年代中后期又重新回到台前。我差不多就是那时候开始读研究生的，大概是1983年，当时有一波持续了大约十年的热潮，然后到九十年代中期兴趣又消退了，一直到2000年代末，我们把它重新包装成了「深度学习」。
04物理学家为何转向研究涌现智能
6:32
Neural net had kind of a bad rep. Um people in computer science and engineering thought neural nets were kind of a bad thing. They had a bad reputation. And so we we branded it into deep learning and sort of brought it back to the fore and then the results were were there in computer vision, in natural language understanding, speech recognition to really convince people that this was uh a good thing. Now, Adam, you at a very young age were interested in theoretical physics, not specifically computer science, and you're watching some of this unfold in some sense from afar. What's the catalyst that sweeps up so many people decades later? There's there's this time where it's of great interest, there's great success in handwriting recognition or in uh uh visual recognition and these things, but it's not sweeping up the world. What what happens that brings us to this point where we're all now talking about large language models?
「神经网络」这个词名声不太好。嗯，计算机科学和工程领域的人觉得神经网络不是什么好东西，它的名声很差。所以我们把它改叫深度学习，让它重新回到台前，然后成果就出来了——在计算机视觉、自然语言理解、语音识别等领域，真正说服了大家这是个好东西。那么Adam，你很小的时候感兴趣的是理论物理，而不是计算机科学，你在某种意义上是从远处看着这一切展开的。是什么契机让几十年后这么多人被卷了进来？曾经有那么一段时间，它引起了很大的兴趣，在手写识别或者视觉识别这些方面取得了很大成功，但它并没有席卷全世界。是什么让我们走到今天这一步，让所有人都在谈论大语言模型？
7:31
So many physicists in the last years have pivoted, should we say, from working on physics to working on AI. And it really traces back to some of the work that Jan and others did to prove that it works. Like when it wasn't working, it was just this this thing that's over there in computer science and like of of many things in the world that are not particularly uh that maybe interesting, but not many physicists are paying attention to it. But then after, you know, Jan and some of the other pioneers of this field proved that it would work, it became a totally fascinating subject for physics. That you link up these neurons together in a certain way and suddenly you get emergent behavior that didn't exist at the individual neuron level. That seems like a a subject that physicists who spend their life imagining how the sort of rich pageantry of the world could emerge from simple laws that immediately attracted the attention of many physicists. And nowadays it's a a a very common career path to do a PhD
过去这些年，非常多的物理学家从做物理转向了做人工智能。而这真的可以追溯到Jan和其他人所做的工作，他们证明了这条路是走得通的。当它还行不通的时候，它就只是计算机科学那边的一个东西，就像世界上很多东西一样，也许有点意思，但没多少物理学家会去关注它。可是后来，你知道，当Jan和这个领域的其他先驱证明了它确实可行之后，它就成了物理学眼中一个特别迷人的课题。你把这些神经元以某种方式连接在一起，突然就出现了在单个神经元层面并不存在的涌现行为。这看起来正是这样一个课题——物理学家一辈子都在设想，世界那种丰富多彩的图景如何能从简单的定律中涌现出来——所以它立刻吸引了很多物理学家的注意。如今，先读一个物理学博士，
05快问快答：AI是否智能、是否有意识
8:29
in physics and then apply it to a emergent system. But the emergent system is an emergent network of neurons that collectively give rise to intelligence. Mhm. Now, let's [clears throat] do a lightning round because you raised the dreaded word intelligence. >> [laughter] >> Um everybody in this room very likely has interacted with something that we're now calling an AI. These are all large language models. And before I ask you to define those for us, I just want to kind of do a lightning round of um of what's your yes or no response to certain things. So um Adam, >> [laughter] >> yes or no, are these AIs, these large language models, uh understanding the meaning of the conversations they're having with us?
然后把它应用到某个涌现系统上，已经是一条非常常见的职业路径。只不过这个涌现系统是一个涌现的神经元网络，它们共同产生了智能。嗯。那么，咱们来做一轮快问快答吧，因为你提到了那个可怕的词——智能。>> [笑声] >> 嗯，在座的每一位很可能都和我们现在称为AI的东西打过交道。这些都是大语言模型。在请你们给它下定义之前，我想先做一轮快问快答，看看你们对一些问题的回答是「是」还是「否」。那么，Adam，>> [笑声] >> 是还是否：这些AI，这些大语言模型，是否理解它们和我们对话的含义？
9:13
Yes or no? Yes. Jan? Sort of. >> [laughter] >> Um perfect. Jan's neurons are not stuck with binary values, is it? >> [laughter] >> Right, exactly. Um it was my fault for giving you a binary choice. Okay, so that allows me to ask the next question cuz it's not a foregone conclusion. If you don't say yes to that, it's going to be interesting what you say to this. Are these AIs conscious? Absolutely not. Adam? Probably not. Okay. [laughter] Um will they soon be? I think they'll one day be conscious if if progress continues in the way that we're we're continuing.
是还是否？是。Jan？算是吧。>> [笑声] >> 嗯，完美。Jan的神经元可不是被困在二值状态里的，是吧？>> [笑声] >> 对，正是如此。是我不好，给了你一个二选一。好，那接下来这个问题我就可以问了，因为它的答案并不是板上钉钉的。如果你对上一个问题不说「是」，那你对这个问题的回答就更有意思了。这些AI有意识吗？绝对没有。Adam？很可能没有。好。[笑声] 嗯，它们很快会有吗？我认为如果进展照现在这样继续下去，总有一天它们会有意识。
9:56
When is hard to say, but One day. Mhm. Yes, for appropriate definitions of consciousness. >> Yes, okay. Well, we do have some philosophers in the house and um we're we're not going to indulge in philosophical definitions of consciousness, or there our hour would go >> [laughter] >> and we'd still be here. Oh, I just heard that groan, I think from our friends up in the balcony. >> [laughter] >> Um but I have one other question. Okay, no, I have two I have two in the lightning round. Uh are we on the precipice of doomsday or a renaissance in human creativity, Jan? Renaissance. Adam?
什么时候很难说，但总有一天会的。嗯。是的，在合适的意识定义之下会有。>> 是的，好。嗯，在座确实有一些哲学家，我们就不去纠缠意识的哲学定义了，否则我们这一小时 >> [笑声] >> 就没了，我们还得坐在这儿。哦，我刚听到有人在叹气，我想是楼上包厢里的朋友们。>> [笑声] >> 嗯，不过我还有一个问题。好吧，不，我还有两个，快问快答里还有两个。我们是站在末日的悬崖边上，还是站在人类创造力复兴的门口，Jan？复兴。Adam？
10:34
Most likely renaissance. >> [laughter] >> Um I have to throw this out the same question to the audience, but I'm going to phrase it more colorfully, which I think they'll relate to. Will the robot overlords rise up against humanity? Yes, hands up. Oh, interesting. Okay, no hands up. Okay, how many robots in the audience? Hands up. Okay. So, okay. So, that's interesting. See, that's cool. It was a little more nose, maybe. Although, the light is blinding. All right, we're going to come back and ask that again at the end.
最有可能是复兴。>> [笑声] >> 嗯，我得把同样的问题抛给观众，不过我会换一个更生动的说法，我觉得大家更有共鸣。机器人霸主会起来反抗人类吗？会的话，请举手。哦，有意思。好，没人举手。好，那观众里有多少个机器人？举手。好。那么，好，这挺有意思的。你看，这很酷。可能刚才「不」的声音多一点。虽然灯光晃得我看不清。好，我们最后还会再问一次这个问题。
06大语言模型如何靠预测下一个词训练
11:11
Um so, here we are. These neural nets have been taught to uh execute a process we now call deep learning and there's other kinds of learning that take off. And what are the large language models specifically, which is really what has swept up the news and people's personal experience? We're we're mostly relating to large language models. And and what are the large language models, Adam? Maybe you could take that. Yeah, so large language model is uh you've probably played with some of them, chat GPT, Gemini made by my company, uh various others um made by other companies.
嗯，那么我们现在的情况是：这些神经网络被训练去执行一个我们现在称为深度学习的过程，还有其他各种学习方式也发展了起来。那么具体来说，大语言模型到底是什么？毕竟真正席卷新闻和大家日常体验的就是它。我们大部分接触到的都是大语言模型。那么大语言模型究竟是什么，Adam？也许你可以来讲讲。好的，大语言模型——你可能已经用过其中一些了，比如ChatGPT、我们公司做的Gemini，还有其他公司做的各种模型。
11:46
It is a special kind of neural network that's trained on particular inputs, on particular outputs, and trained in a particular way. So, it is at At heart, it is mainly the kind of deep neural network that was pioneered by by Jan and by others, but uh with a particular architecture designed for the following task. Uh it takes text in, so it'll it'll read some uh the first few words of some sentence or the first few paragraphs of some book, and it will try and predict what the next word is going to be.
它是一种特殊的神经网络，用特定的输入、特定的输出，并以特定的方式来训练。所以它本质上主要就是Jan和其他人开创的那种深度神经网络，只不过采用了一种为下面这个任务专门设计的架构。它接收文本输入，也就是它会读一句话的前几个词，或者一本书的前几段，然后试着预测下一个词会是什么。
12:20
And so, you take a deep neural network with a particular architecture, and you have it read, basically to first approximation, the entire internet. And for every word that comes along on the entire internet, all of the text data and all other kind of data you can find, uh you then ask it, "What do you think the next word's going to be? What do you think the next word's going to be?" And to the extent that it it gets it right, you give it a little bit of uh reward and strengthen those neural pathways. To the extent that it gets it wrong, you you diminish those neural pathways. And if you do that, uh it'll just start off spewing just completely random words for its prediction, but uh if you train it on a million words, it'll still be spewing random words. If you train it on a a billion words, it'll maybe have just started to learn subject-verb-object and various bits of sentence structure. Uh and if you train it, as we do today, on on a trillion words or more, tens of trillions of words, uh then it'll start become the
所以，你拿一个特定架构的深度神经网络，让它去读——粗略来说——整个互联网。互联网上出现的每一个词，所有你能找到的文本数据和各种其他数据，你都问它：「你觉得下一个词会是什么？你觉得下一个词会是什么？」如果它答对了，你就给它一点奖励，强化那些神经通路。如果它答错了，你就削弱那些神经通路。这么做的话，一开始它只会吐出完全随机的词作为预测；如果你用一百万个词训练它，它还是在乱吐词。如果你用十亿个词训练它，它可能刚开始学会主谓宾以及一些句子结构。而如果你像我们今天这样，用一万亿个词甚至更多、几十万亿个词来训练它，那它就会开始变成那个你今天大概——我希望——已经玩过的对话伙伴。
07文本训练量与四岁孩子感官数据之比
13:19
conversation partner that you you probably, I hope, uh played around with today. No, um it it it strikes me as intriguing, like it's it's it amuses me sometimes people get really outraged at their chatbot that they're engaged with when it leads them astray or lies to them. And sometimes I've said, "Well, it's it's doesn't need to be words. It it might as well be colors or symbols. It's just playing a mathematical game, and therefore doesn't have a sense of meaning." Now, I know Adam sort of objected to my summary of that.
嗯，这让我觉得很有意思，有时候人们会对自己正在用的聊天机器人特别气愤，因为它把人带偏了或者对人撒谎。我有时候会说：「其实它处理的不一定得是词，也可以是颜色或者符号。它只是在玩一个数学游戏，因此并没有意义感。」不过我知道Adam对我这个概括是有异议的。
13:54
Do you think that they are extracting meaning um in the same sense that we do when we are engaging in composing sentences? Well, they're certainly extracting some meaning, um but it's it's a lot more superficial than what most humans uh would extract from from text. Most humans uh intelligence is linked to is is grounded into underlying reality, right? And language is a way to express phenomena or things in that or concepts grounded in that reality. Um and LLMs don't have any notion of the underlying reality. And so, their understanding is is relatively superficial.
你认为它们提取到的「意义」，和我们在组织句子时所理解的意义是一回事吗？嗯，它们肯定提取到了某种意义，但这比大多数人从文本中提取到的东西要肤浅得多。大多数人的智能是与底层现实相连、扎根于底层现实的，对吧？而语言是一种表达那个现实中的现象、事物或概念的方式。而大语言模型对这个底层现实没有任何概念。所以它们的理解是相对肤浅的。
14:43
Um they don't really have common sense in the in the way that we understand it. Um but if you train them long enough, they they will answer correctly most questions that people will think about asking. That's the way they're trained. You you you collect all the questions that everybody has ever asked them, and then you train them to produce the correct answer for this. Now, there's always going to be new questions or new prompts, new sequences of words for which the system has not really been trained, and for which it might produce complete nonsense. Okay, so in that sense, they don't have the real understanding of the underlying reality, or they do have an understanding, but it's it's superficial.
嗯，它们并不真正具备我们所理解的那种常识。嗯，但如果你训练它们足够久，它们对人们想得到的大多数问题都能答对。它们就是这么训练出来的。你把所有人问过的问题都收集起来，然后训练它们给出正确的答案。那么，总会出现新的问题、新的提示词、新的词语序列，是系统没有真正被训练过的，对这些它可能会输出完全是胡说八道的东西。好的，所以从这个意义上说，它们对底层现实并没有真正的理解，或者说它们是有理解的，但那种理解很浅薄。
15:26
Um and so, you know, the next question is, how do we fix that? So, I I could play devil's advocate and say, "Well, how do I know that what a human being doing is doing is that much different?" Right? We're trained on lots of language. We get some dopamine hit or some reward system for having said the right word at the right time and the right grammatical structure for the language that we're immersed in. And um and we back propagate. >> [laughter] >> We try to do a better job the next time. In some sense, how how is that different uh than what a human being is doing? And you're you're saying maybe it's the sensory experience of being immersed in the world?
嗯，所以你知道，下一个问题就是：我们怎么解决这个问题？我可以唱个反调，说："那我怎么知道人类在做的事情就有多大不同呢？"对吧？我们也是在大量语言上被训练的。我们在恰当的时机说出恰当的词、用上我们所浸润的那门语言的正确语法结构时，会得到某种多巴胺的刺激或者某种奖励，嗯，然后我们做反向传播。>> [笑声] >> 我们下次会努力做得更好一点。从某种意义上说，这跟人类在做的事情有什么区别呢？你的意思是，也许区别在于沉浸在这个世界中的那种感官体验？
16:06
Okay. Um a typical LLM, as Adam mentioned, is trained on tens of trillions of of words, typically It's only a few hundred thousand words, I think. You're just saying sentences. It's combinations. >> 30 trillion 30 trillion words is is a typical size for the training set pre-training of of an LLM. Uh a A word is represented actually as sequences of tokens, doesn't really matter. Uh and a token is about 3 bytes, so the total is about 10 to the 14 bytes, right? One with 14 zeros um of training data to train those LLMs.
好的。嗯，正如 Adam 提到的，一个典型的大语言模型是在几十万亿个词上训练的，通常……人类大概只有几十万个词吧，我想。你只是在说句子。是各种组合。>> 30 万亿。30 万亿个词是大语言模型预训练数据集的典型规模。呃，一个词实际上是用一串 token 来表示的，这其实无所谓。呃，一个 token 大约是 3 个字节，所以总量大约是 10 的 14 次方字节，对吧？就是 1 后面跟 14 个零，这么多训练数据来训练那些大语言模型。
16:47
And that corresponds to basically all the text that is uh publicly available on the internet plus some other stuff. And it would take any of us something like half a million half a million years for any of us to read through that material, right? So, it's an enormous amount of textual data. Now, compare this with what a child uh perceives uh during the first few years of life. Um psychologists tell us that a 4-year-old has been awake a total of 16,000 hours. Um and there's about 1 byte per second going through the optic nerve, every single fiber of the optic nerve, and we have 2 millions of them. So, it's about 2 megabytes per second getting to the visual cortex.
这基本上相当于互联网上所有公开可得的文本，再加上一些别的东西。而我们任何人要读完这些材料，大概需要五十万年，五十万年，对吧？所以这是极其庞大的文本数据量。那么，把这个跟一个孩子在生命最初几年里所感知到的东西比较一下。嗯，心理学家告诉我们，一个 4 岁孩子清醒的时间总共是 16,000 小时。嗯，视神经每一根纤维每秒大约传输 1 个字节的信息，而我们有 200 万根这样的纤维。所以每秒大约有 2 兆字节的信息进入视觉皮层。
17:32
Um During 16,000 hours, do the arithmetics, and it's about 10 to the 14 bytes. A 4-year-old has seen as much visual data as the biggest LLM trained on the entire text ever produced. And so, what that tells you is that there is way more um information in the real world, but it's also much more complicated. It's noisy, it's high-dimensional, it's continuous, and basically the methods that are employed to train LLMs do not work in the real world. That explains why we have LLMs that can pass the bar exam or solve equations or compute integrals like college students and solve math problems, but we still don't have a domestic robot that can, you know, do the chores in the house. We don't we don't even have level five self-driving cars. I mean, we have them, but we cheat.
嗯，在 16,000 小时里，算一算，大约就是 10 的 14 次方字节。一个 4 岁孩子所看到的视觉数据量，跟在人类有史以来产生的全部文本上训练的最大的大语言模型所用的数据量一样多。所以这告诉你的是，真实世界里包含的信息要多得多，但它也复杂得多。它有噪声、是高维的、是连续的，而且基本上，训练大语言模型所用的那些方法在真实世界里是行不通的。这就解释了为什么我们有能通过律师资格考试、能像大学生一样解方程、算积分、做数学题的大语言模型，但我们仍然没有一个家用机器人能，你知道的，做家务。我们连 L5 级自动驾驶汽车都还没有。我是说，我们有，但我们是在作弊。
08样本效率之争：猫、棋手与奥数金牌
18:26
So, um I mean, we certainly don't have self-driving cars that can learn to drive in 20 hours of practice like any teenager, right? So, obviously, we're missing something very big to get machines to the level of human or even animal intelligence. Well, let's not talk about language. Let's talk about how a cat is intelligent or dog. Um we're not even at that level with AI systems. Adam, you you you impart more comprehension on uh the part of the LLMs at this point already. Uh I think that's right. So, I mean, Jan is making sort of excellent points that the LLMs are much less, for example, sample efficient than humans. Humans or indeed your cat or just a a cat, I don't know if it was your cat or any cat in your >> cat.
所以，嗯，我们肯定还没有能像任何青少年那样、练 20 个小时就学会开车的自动驾驶汽车，对吧？所以显然，要让机器达到人类甚至动物的智能水平，我们还缺了非常重要的东西。好吧，我们先别谈语言。我们来谈谈猫或者狗是怎么个聪明法。嗯，AI 系统连那个水平都还没达到。Adam，你倒是认为现阶段的大语言模型已经具备更多的理解能力了。呃，我觉得他说得对。我是说，Yann 提出了非常好的观点：比如说，大语言模型的样本效率比人类低得多。人类，或者说你的猫，或者随便哪只猫——我不知道你举的例子里是你的猫还是随便一只猫——>> 猫。
19:18
>> in your example, um is able to learn from many fewer examples than a large language model, for example, can learn from. It takes way more data to teach it to the same level of proficiency. Um And and that's true, and that that is a thing that is better about uh the, you know, architecture of animal minds compared to these artificial minds that we're building. Um on the other hand, sample efficiency isn't everything. Um we see this frequently, in fact, when we try and, you know, before large language models, when we and put uh machines on you make artificial minds to do other tasks, even the famous chess bots that we built on built on top of large language models, the way they were trained sort of Alpha Zero and various other ones, they would play each other they would play itself at chess a huge number of times and to begin with it would just be making random moves and then every time it won or lost the game when it was playing itself it was sort of uh reward that neural pathway or punish
>> 在你的例子里，嗯，它能从远比大语言模型少得多的样本里学会东西。要把大语言模型教到同样的熟练程度，需要多得多的数据。嗯，这是事实，而且这确实是动物大脑的架构相比我们正在造的这些人工心智更优越的地方。嗯，但另一方面，样本效率并不是一切。嗯，我们经常看到这一点。事实上，当我们尝试——你知道，在大语言模型出现之前，当我们让机器、让人工心智去做别的任务时，甚至是那些著名的下棋机器人，我们在……在大语言模型之上构建的，它们的训练方式，就像 AlphaZero 和其他几个一样，它们会自己跟自己下无数盘棋，一开始它只是在随机走子，然后每当它自己跟自己下赢了或者输了一盘棋，就会强化或者惩罚相应的神经通路，然后它们再一次次地对弈。当它们下的棋局数量
20:22
that neural pathway and they play each other at chess again and again and when they played as many games as a human grandmaster has played, they were still making essentially random moves. But they didn't just we're not confined to making the same number of playing the same number of games that a human grandmaster could play because silicon chips are so fast, because we can build them with such parallel processing, they were able to play many more human many more games than any human could ever play in their lifetime.
跟一位人类特级大师一生下过的棋局数量相当时，它们仍然基本上是在随机走子。但它们并不局限于只能下人类特级大师那么多盘棋，因为硅芯片实在太快了，因为我们能造出高度并行处理的芯片，它们能下的棋局远远超过任何人类一辈子能下的数量。
20:51
And what we found is that when they did that, they reached and then far surpassed the level of human chess players. They're less sample efficient but that doesn't mean they're worse at chess. It is clear that they're much better at chess. So too with understanding. When we it is it is true that we can you know it is harder with these things to you need more samples to get them up to the same level of proficiency but the question is once they've reached that, can we use the fact that they are so much more general and so much more so much faster and more inherent to push beyond that. I mean another example perhaps with the cat is a cat is in fact even more sample efficient than a human.
而我们发现，当它们这么做之后，它们达到了、随后又远远超越了人类棋手的水平。它们的样本效率是更低，但这并不意味着它们下棋更差。很明显它们下棋厉害得多。理解这件事也是同理。当我们……确实，用这些东西确实更难，你需要更多的样本才能把它们提升到那个同样的熟练程度，但问题是，一旦它们达到了那个水平，我们能不能利用它们通用性强得多、速度快得多、也更内在的这一事实，把它们推向更远的地方。我是说，另一个例子也许是猫——猫其实比人类的样本效率还要高。
21:32
A human takes a a year to learn to to walk. A cat learns to walk in a in a week or so. You know, it's much faster. That does not mean that a cat is smarter than a human. It does not mean that a cat is smarter than a large language model. The final question at the end should be what is the capabilities of these things. How far can we push the capabilities and on almost every except for the somewhat impoverished metric of sample efficiency, on every metric that counts, we've pushed these large language models far beyond the frontier of cat intelligence. So >> [laughter] [gasps] >> Yes.
人类要花一年才学会走路，而猫大概一周就学会走路了。你知道，快得多。但这并不意味着猫比人类更聪明，也不意味着猫比大语言模型更聪明。最终的问题应该是：这些东西的能力到底有多强？我们能把它们的能力推到多远？而在几乎所有指标上——除了样本效率这个多少有点贫瘠的指标之外——在每一个真正重要的指标上，我们已经把这些大语言模型推得远远超过了猫的智能水平。所以 >> [笑声] [惊呼] >> 是的。
22:11
I don't understand why we're not making cats but >> [laughter] >> Sorry, what was I on? I mean certainly the LLMs in question have much more accumulated knowledge than cats or even humans for that matter. And we do have many examples of computers being far superior to humans in a number of you know different tasks like playing chess for example. That's something I mean it just means that humans just suck at chess. That's all it means. No, we really suck at chess and go by the way even even more. And and many other tasks that computers are much better than than us um at at at solving. Um so certainly LLMs can accumulate huge amount of of of of knowledge and some form of them can be trained to translate languages, understand spoken language and and translate it into another one from you know a dozen languages to another dozen languages in any direction. No human can do this.
我不明白我们为什么不去造猫，不过 >> [笑声] >> 抱歉，我刚说到哪了？我是说，这些大语言模型确实拥有比猫、甚至比人类更多的积累知识。而且我们确实有很多例子，计算机在不少不同任务上远远优于人类，比如下国际象棋。那件事我是说，那只不过说明人类下棋很烂罢了。它就只说明这一点。不，我们下国际象棋确实很烂，顺便说一句，下围棋更烂。还有很多其他任务，计算机解决起来都比我们强得多。嗯，所以大语言模型确实能积累海量的知识，而且其中某些形式可以被训练来做语言翻译、理解口语并把它翻译成另一种语言——从十几种语言译到另外十几种语言，任意方向都行。没有人类能做到这一点。
23:11
Um so they they do have superhuman capabilities but the ability to learn quickly, efficiently, to apprehend a new problem that we've never been trained to solve and be able to come up with a solution um and to really you know understand a lot about how the how the world behaves that is still out of reach of AI systems at the moment. Mhm. I I would I mean we've had recent successes with this where it is not the case that they're just taking problems that they've seen before letter for letter and looking up the answer in a in a lookup table or even that they're uh they are they are in some sense doing pattern matching but they're doing pattern matching at a sufficiently elevated level of abstraction that they're able to do things that if they've never seen before and no no human can do. So there's a there's a competition uh each year called the International Math Olympiad.
嗯，所以它们确实具备超人的能力，但快速、高效学习的能力，理解一个我们从未被训练去解决的新问题并能想出解决方案的能力，以及真正理解这个世界如何运作的很多东西——这些目前仍然是 AI 系统做不到的。嗯。我我想说，我是说我们最近在这方面取得了一些成果，情况并不是它们只是把以前见过的、一字不差的问题拿来，在一个查找表里查答案，甚至也不是说呃它们在某种意义上是在做模式匹配——但它们做模式匹配的抽象层级足够高，以至于它们能做到一些以前从未见过、而且没有任何人类能做到的事情。所以有一个比赛，呃每年举办一次，叫国际数学奥林匹克。
24:06
Um it is the very smartest uh finishing high school maths uh teenagers in the entire world. They're all given six problems uh each year. The pinnacle of human intelligence. I have some mathematical ability. I look at these problems. I don't even know where to start. Um you know, this this year we fed them into our machine uh as as did a number of other LLM companies and they took these problems they'd never seen before. They were completely fresh, didn't appear anywhere in the training data, completely made up, took a whole bunch of different ideas, combined the different ideas and got a score on these tests that was better than all except the first dozen the top dozen humans on the planet. I think that's that's pretty impressive intelligence. Mhm.
嗯，参赛的是全世界最聪明的、即将高中毕业的数学少年。他们每年都会拿到六道题。这是人类智力的巅峰。我自己有一些数学能力，我看着这些题，我甚至不知道该从哪儿下手。嗯，你知道，今年我们把这些题喂给了我们的模型，呃其他一些大语言模型公司也这么做了，模型拿到这些从未见过的题目，题目是全新的，训练数据里任何地方都没出现过，完全是新编的，它调用了一大堆不同的思路，把这些思路组合起来，在这些测试上取得的分数超过了地球上除了前十几名之外的所有人类。我觉得这算是相当令人印象深刻的智能了。嗯。
09机器是否真的理解？可解释性证据
24:51
I I guess the question is um back to this idea do they understand? Do you we can look at the mathematics of the model. There's some input data. We understand what it's doing. It is a black box which is kind of fascinating. It's just so complex that it's not as though we can't do that with the human mind either. It's not as though you can look at the inner workings and and see exactly what they're doing. To some extent it is a black box but we presume it's just doing these calculations. It's moving in matrices. It's working in some vector space. It's doing some higher dimensional thing. I have some experience of understanding. I guess people are still grasping at that. Is it having some experience of understanding?
我想问题是,回到这个想法——它们真的理解吗?我们可以去看这个模型的数学原理。有一些输入数据。我们知道它在做什么。它是个黑箱,这一点挺让人着迷的。它只是太复杂了,但这也不是说我们对人脑就能做到,并不是说你可以去看它内部的运作机制,然后准确地看出它们在做什么。某种程度上它是个黑箱,但我们假定它只是在做这些计算。它在矩阵里运算,在某个向量空间里工作,在做某种高维度的事情。而我有一种理解的体验。我猜大家还在琢磨这个问题:它是否也有某种理解的体验?
25:32
Is it important whether or not they experience understanding? Is that sufficient to call it comprehension of meaning? Are you describing understanding as a behavioral trait here where it gives the right answers to problems or whether it deeply at the neural level understands? Yeah, I'm I'm completely at the whims of the philosophers here. No, I I don't know if I understand that at my at the human level, right? I can't tell you what process I'm executing at the moment either, right? But I do have some intuitive subjective experience that I understand the conversation. Obviously not that well. Um but but uh I when I'm talking to you, I feel you are understanding and when I'm talking to chat GPT, I do not. And you're telling me I'm mistaken.
它们是否体验到理解,这重要吗?这足以称之为对意义的领会吗?你这里说的理解是指一种行为特征吗?也就是它能给出问题的正确答案;还是说它在神经层面上深刻地理解?是啊,我在这方面完全听凭哲学家们摆布。不,我不知道我在人类层面上是否理解这个,对吧?我也说不出我此刻正在执行什么过程,对吧?但我确实有某种直觉上的主观体验,觉得自己理解这段对话。显然理解得没那么好。嗯,但是当我跟你说话时,我感觉你是在理解;而当我跟 ChatGPT 说话时,我不觉得。而你在告诉我,我搞错了。
26:21
It's understanding as well as I am or you are. In my opinion, it is understanding, yes. And I think there's two different pieces of evidence for that. One is I think if you talk to them like if you talk to them and ask them about difficult concepts I'm frequently surprised and with every passing month and every new model that comes out, I am more and more surprised at the level of sophistication with which they're able to discuss things. And so just just at that level, it it's super impressive. I I would really encourage everybody here to talk to these large language models if you've not already. You know, when the science fiction writers imagined that we'd built some sort of Turing test passing machine that that was going to you know some new alien intelligence that we'd have in a box, they all imagined that we'd sort of hide it in a basement, you know, in a castle surrounded by a moat with armed guards and we'd only have like a priestly class who could be able to go and talk to it. That is not not it's not
它的理解程度跟我或者跟你一样。在我看来,它确实在理解,是的。我认为有两方面不同的证据。一是,我觉得如果你跟它们对话,比如跟它们聊,问它们一些困难的概念,我常常感到惊讶;而且随着每个月过去、每个新模型发布,我越来越惊讶于它们讨论问题时所展现出的精细复杂程度。所以单就这一点而言,就已经非常令人印象深刻了。我真的很鼓励在座各位去跟这些大语言模型聊一聊,如果你还没试过的话。你知道,当科幻作家们设想我们造出某种能通过图灵测试的机器,某种我们关在盒子里的新型外星智能时,他们都想象我们会把它藏在地下室里,你知道,在一座被护城河和武装守卫环绕的城堡里,而且只有类似祭司阶层的人才能去跟它对话。结果并不是这样,事情不是这么发展的。实际的
27:23
the way it worked out. The way it's worked out is the first thing we did is we immediately hooked it up to the internet and now anybody can go talk to it and I would highly encourage you to to talk to these things and explore in areas that you know to see both their limitations but also their strength and their their depth of understanding. So I'd say that's the first piece of evidence. The second piece of evidence is you said they're a black box. They're not exactly a black box. We do have access to their neurons. In fact, we have much better access to the neurons of these things than we do with a human.
情况是,我们做的第一件事就是立刻把它接上互联网,现在任何人都可以去跟它对话。我强烈建议你去跟这些东西聊聊,在你熟悉的领域里探索一下,既看看它们的局限,也看看它们的长处和理解的深度。所以我想说这是第一方面的证据。第二方面的证据是,你说它们是黑箱。它们并不完全是黑箱。我们确实能接触到它们的神经元。事实上,我们对这些东西的神经元的接触程度,比对人类的要好得多。
27:52
It's very hard to get IRB approval to slice up a human while they're doing a math test and see how their neurons are firing and if you do do that, you can only do that once on on a per human basis. Whereas these neural networks, we can freeze them, replay them, write down everything that happened. If we're curious, we can go and go and prod their neurons in certain ways and see what happened. And so this is it's still rudimentary but this is the field of interpretability, mechanistic interpretability, trying to understand not just what they say but why they say it, how they think it.
要拿到伦理审查委员会的批准,在一个人做数学题的时候把他切开、看看他的神经元怎么开火，如果你真要那么做，每个人也只能做一次。而这些神经网络，我们可以把它们冻结、重放，把发生的一切都记录下来。如果我们好奇，还可以用各种方式去戳一戳它们的神经元，看看会发生什么。所以这仍然很初级，但这就是可解释性、机制可解释性这个领域，它试图理解的不只是它们说了什么，而是它们为什么这么说、它们是怎么想的。
28:23
And when you do that, we see uh when you feed them a math problem problem, there's a little bit of a a circuit there that computes the answer that that we didn't program it to have that. It learned how to do that. While trying to predict the next token on all of this text, it learned that in order to most accurately predict the next the next word, I should say. In order to most accurately predict the next word, it needed to figure out uh how to do maths and it needed to build a sort of proto little circuit inside it to do the mathematical computations.
当你这么做的时候，我们会看到，呃，当你喂给它们一道数学题时，里面会有一小段电路在计算答案，而这是我们并没有编程让它拥有的。它是自己学会怎么做的。在学习预测所有这些文本的下一个 token 的过程中，它学到了：为了最准确地预测下一个——下一个词，我应该说。为了最准确地预测下一个词，它需要弄明白，呃，怎么做数学运算，还需要在它内部构建出某种原始的小电路来完成这些数学计算。
10机器学习很烂：像素预测失败与JEPA构想
28:53
Mhm. Now, Ian, you famously threw a slide up at one of your uh keynote lectures that was very provocative, um very scholarly. It said um machine learning sucks, I believe was it. And then that kind of went wild. Ian LeCun says machine learning sucks. Um why are you saying machine learning sucks? Adam has just told us how phenomenal it is. He talks to them >> [laughter] >> and wants us to do the same. Um why do you think it sucks? What's the problem? >> [clears throat] >> Well that statement has been widely misinterpreted but the point the point I was making is the point that uh we both we both made which is that why is it that a teenager can learn to drive a car in 20 hours of practice uh a 10-year-old can clean up the dinner table and fill up the dishwasher the first time you ask the child to do it. Whether the 10-year-old will want to do it is a different story, but you know, certainly can. Um we don't have robots that are anywhere near this.
嗯。那么，Yann，你在一次主题演讲上打出过一张很出名的幻灯片，非常有挑衅性，呃，又很有学理。上面写着，呃，机器学习很烂，我记得是这样吧？然后那就火了。Yann LeCun 说机器学习很烂。呃，你为什么说机器学习很烂？Adam 刚刚才告诉我们它有多惊人。他还跟它们聊天 >> [笑声]>> 而且想让我们也这么做。呃，你为什么觉得它很烂？问题出在哪儿？>> [清嗓子] >> 嗯，那句话被广泛误解了，但我当时想表达的意思是我们俩都提到过的一点是，为什么一个十几岁的孩子能在20个小时的练习里学会开车一个10岁的孩子第一次被你叫去收拾餐桌、把碗碟放进洗碗机就能做到。至于这个10岁的孩子愿不愿意做，那是另一回事，但他肯定是能做到的。而我们现在的机器人根本达不到这种水平。
29:59
And we don't have robots that are even anywhere near the you know, physical understanding of of reality of of a cat or a dog. And so in that sense, machine learning sucks. It doesn't mean that the the deep learning method, the back propagation algorithm, the neural nets suck. That was obviously excellent. Yes. >> Yes, obviously. That's great. And [laughter] we don't have any alternative to this. And uh I I certainly believe that you know, neural nets and deep learning and back propagation will be you know, are with us for for a long time. We'll be the basis of uh future AI systems. But but how is it that uh uh you know, young humans can can learn how the world works in the first few months of life?
我们的机器人甚至连猫或狗那种对物理现实的理解都远远达不到。所以从这个意义上说，机器学习很糟糕。这并不是说深度学习方法、反向传播算法、神经网络很糟糕。那些显然非常出色。是的。>> 是的，显然如此。那些很棒。[笑声] 而且我们也没有别的替代方案。我确实相信神经网络、深度学习和反向传播还会伴随我们很长时间。它们会成为未来AI系统的基础。但问题是，年轻的人类是怎么在生命最初的几个月里就学会这个世界是怎么运作的？
30:43
It takes 9 months for a human babies to learn um intuitive physics like gravity, inertia, and things like this. Um baby animals learn this much faster. They have smaller brains, so it's easier for them to learn. Um They don't learn to the same level, but they but they do learn faster. And and so, you know, there is this type of learning that we need to reproduce. Um and we'll do this with back prop, with neural nets, with deep learning. It's just that we're missing a concept, an architecture. Um so I've been I've I've been making proposals for the type of architectures that could possibly uh learn this kind of stuff. You know, why is it that LLMs handle language uh so easily? It's because um as Adam described, you you train an LLM to predict the next word or the next token, doesn't matter.
人类婴儿要花9个月才能学会直觉物理，比如重力、惯性这些东西。动物幼崽学这些要快得多。它们的大脑更小，所以学起来更容易。它们学到的水平不一样，但确实学得更快。所以你看，存在这样一类需要学习并复现这些能力。嗯，我们会用反向传播、用神经网络、用深度学习来做这件事。只是我们还缺一个概念，一种架构。嗯，所以我一直在提出一些架构方案，看看什么样的架构有可能学会这类东西。你知道吗，为什么大语言模型处理语言这么轻松？这是因为，嗯，正如 Adam 描述的那样，你训练一个大语言模型去预测下一个词或者下一个 token，都无所谓。
31:33
There's only a finite number of words in the dictionary. So you can never actually predict exactly which word comes after a sequence. But you can train a system to produce essentially what amounts to a score for every possible words in your dictionary or a probability distribution over every possible words. So essentially what an LLM does is that it produces a long list of numbers between zero and one that sum to one, which for each word in the dictionary says, this is the likelihood that this word appears right now. You can represent the uncertainty in the prediction this way. Now, try to translate it um the the same principle. Instead of training a system to predict the next word, um feed it with a video and ask it to predict what happened next in the video.
词典里的词是有限的。所以你其实永远没法准确预测某个序列之后到底是哪个词。但你可以训练一个系统，为你词典里的每一个可能的词给出一个分数，或者说给出一个覆盖所有可能词的概率分布。所以本质上，大语言模型做的事就是产生一长串介于 0 和 1 之间、加起来等于 1 的数字，对词典里的每个词来说，它表示的是这个词此刻出现的可能性有多大。你可以用这种方式来表示预测中的不确定性。那现在，试着把嗯，同样的原理迁移过去。不是训练系统去预测下一个词，嗯，而是给它一段视频，让它预测视频里接下来发生了什么。
32:16
And this doesn't work. I've been trying to do this for 20 years. And it it really doesn't work if you try to predict at a pixel level. Uh and it's because the real world is messy. There's a lot of things that that may happen, plausible things that may happen. Um and you can't really represent a distribution over all possible uh things that may happen in the future because it's basically an infinite list of possibilities and we don't know how to represent this um efficiently. And so those those techniques that work really well for text or for sequences of of symbols do not work for real world sensory data.
这行不通。我已经试了二十年了。如果你想在像素层面做预测，那是真的行不通。呃，这是因为真实世界是混乱的。有太多可能发生的事情，有太多貌似合理、可能会发生的情况。嗯，你没办法真的去表示一个覆盖未来所有可能情况的分布，因为那基本上是一个无穷的可能性列表，而我们不知道如何高效地表示它。所以那些对文本、对符号序列非常有效的技术，用在真实世界的感官数据上就不管用了。
11LLM是否已饱和：符号推理易，物理直觉难
32:57
They just don't They absolutely don't. And and so we need to invent new techniques. So one of the things I've been proposing is one in which the the system learns an abstract representation of what it observes and it makes prediction in that abstract representation space. And this is really the way humans and animals function. And we we find abstractions that allow us to make predictions while ignoring all the detail the details we cannot predict. So you really think that despite the phenomenal successes of these LLMs that they are limited and and their limit is quickly approaching. You don't think that they're scalable to this you know, artificial general intelligence or super intelligence.
就是不行。绝对不行。所以我们需要发明新的技术。我一直提议的方案之一是：让系统学习一种关于它所观察到的东西的抽象表示，然后在那个抽象表示空间里做预测。这其实正是人类和动物运作的方式。而且我们会找到一些抽象层次，让我们能够做出预测，同时忽略掉所有我们无法预测的细节。所以你真的认为，尽管这些大语言模型取得了惊人的成功，它们仍然是有局限的，而且这个上限很快就要到了。你不认为它们能扩展到你说的那种通用人工智能或者超级智能。
33:37
>> That's right. No, they don't. [clears throat] And and in fact, we see the performance saturating. So we we see uh progress in in some domains like mathematics, for example. And mathematics and and code generation, you know, programming are two domains where the uh the the manipulation of symbols actually gives you something, right? As a physicist, you you know this, right? You write the equation and it actually kind of You follow it. You you can follow it and it it's uh it drives your your thinking into some extent, right? I mean, you you drive it by intuition, but but the symbol manipulation itself actually has uh meaning. So this type of problems, LLMs actually can handle pretty well, where the the reasoning really consists in kind of searching through sequences of symbols. But it's only there's only a small number of problems for which that's the case.
>> 没错。不，它们做不到。[清嗓子] 而且事实上，我们已经看到性能在饱和了。所以我们看到，呃，在某些领域是有进展的，比如数学。而数学和代码生成，你知道的，编程，这两个领域里呃，符号操作本身确实能给你带来一些东西，对吧？作为物理学家，你懂这个的，对吧？你写下方程，它就真的会……你顺着它走。你可以顺着它推下去，呃，它在某种程度上引导着你的思考，对吧？我是说，你是靠直觉在驱动它，但符号操作本身其实是有意义的。所以这类问题，大语言模型其实处理得相当好，也就是那种推理本质上就是在符号序列里做搜索。但这样的问题只占很小一部分。只有很少一类问题是这样的。
34:25
Chess playing is another one. Um you search through sequences of of of moves that, you know, for a good one or sequences of uh derivations in mathematics that will produce a particular result, right? Um but in the real world, it you know, in high-dimensional continuous things where the search has to do with like how do I move my muscles to uh you know, grab this uh you know, grab grab this um this this glass here. I'm not going to do it with my left hand, right? I'm going to have to change hand with this and and then grab it, right? You need to plan and have some understanding of what's possible, what's not possible, that, you know, I can't just attract the glass, you know, by telekinesis or I can't just I can't just make it appear in my in my left hand like this. I can't have my hand kind of cross my body. Like you know, all of those intuitive things, we we learn them when we were babies. Um and and we learn, you know, how our body reacts to our controls and how uh you know, the the world reacts to to the actions we
下棋是另一个例子。你在一步步的走法序列里搜索，找出好的走法；或者在数学里搜索一系列推导，以得出某个特定结果，对吧？但在真实世界里，在那种高维连续的场景中，搜索要解决的是比如：我该怎么运动我的肌肉，去把这个……你知道，��抓住这个杯子。我不会用左手去拿，对吧？我得换只手，然后再抓住它，对吧？你需要规划，需要对什么可行、什么不可行有一定理解，比如说，我没法靠意念把杯子吸过来，我也没法让它凭空出现在我左手里，就这样。我的手也没法穿过我的身体。所有这些直觉性的东西，我们都是在婴儿时期学会的。而且我们学会了自己的身体如何响应我们的控制指令，以及这个世界如何对我们采取的行动做出反应。所以，如果我推这个杯子，我知道它会滑动。如果我
35:31
take. So so, you know, if I push this glass, I know it's going to slide. If I push it from the top, maybe it maybe it's going to flip. Maybe not because the friction is not that high. If I push with the same force on this table, it's not going to flip. You know, we have all those those intuitions that allow us to kind of apprehend the real world. Uh but this is it turns out much much more complicated than manipulating language. We think of language as kind of the epitome of you know, human intelligence and stuff like that. It's actually not true.
从上面推它，也许它会翻倒。也许不会，因为摩擦力没那么大。如果我用同样的力气推这张桌子，桌子是不会翻的。我们有所有这些直觉，让我们能够去把握真实世界。但事实证明，这比操纵语言要复杂得多得多。我们总把语言当成人类智能的巅峰之类的东西，其实并非如此。
12五年能力狂飙与递归自我改进的可能
36:01
Language is actually easy. >> [laughter] >> Is it the Moravec paradox that what computers are good at, humans are bad at? What humans are good at, computers are bad at? Yeah, we keep running into the Moravec paradox here. Now, Adam, I I know that you are less pessimistic about the potential of the current neural net deep learning um paradigm and you see the potential for a great escalation in success and uh you don't see it saturating. Um what's your thought about that? >> I am I don't. That's right. Um And so yeah.
语言其实是简单的。>> [笑声] >> 这是不是就是莫拉维克悖论——计算机擅长的，人类不擅长；人类擅长的，计算机不擅长？是的，我们在这里一次次撞上莫拉维克悖论。那么，Adam，我知道你对当前神经网络深度学习范式的潜力没那么悲观，你认为它有可能带来成功的巨大跃升，而且你不认为它会趋于饱和。你对此怎么看？>> 我确实不认为。没错。嗯，是这样。
36:39
>> witnessed over the last 5 years the most extraordinary run-up in capabilities in any system I've ever seen. This is what transfixed my attention. It's what transfixed many other people uh in AI and neighboring fields to focus all of our attention on this matter. I don't see any slowdown in the capabilities. A year ago if you just look at all of the all of the metrics we used to judge how good these large language models are, they're getting stronger and stronger and stronger. Things that they, you know, a year the model from a year ago today would be, you know, table stakes, would be considered extremely poor. Every few months these things push their capabilities. And if if you track their capabilities on all of these tasks, they're heading towards superhuman on on almost all of them. It's already better gives better legal advice than uh than a lawyer. It gives better um it's a better poet than almost every poet you will come. In my little in my little area of physics, uh I I use it because like there's
>> 过去五年里，我见证了我所见过的任何系统中最不可思议的能力飙升。这是让我全神贯注的事情。它也吸引了很多其他人的注意力——AI 界和相邻领域的人，让我们把全部注意力都放到这件事上。我没有看到能力提升有任何放缓。一年前，如果你去看我们用来评判这些大语言模型有多好的所有指标，它们在变得越来越强、越来越强。那些一年前的模型今天能做到的事情，如今已经是基本门槛了，会被认为差得离谱。每隔几个月，这些东西就把自己的能力往前推一步。如果你追踪它们在所有这些任务上的能力，它们几乎在所有任务上都在朝着超越人类的方向走。它现在给出的法律建议已经比律师更好了。它写诗也比你能遇到的几乎所有诗人都更好。在我自己所在的物理学的小领域里，我会用它，因为总有些东西我按理应该知道，但我不知道。我会去问语言模型，
37:50
something I kind of should know, but I don't. I'll ask the language model and it will not only tell me what the right answer is, it will patiently and I should say non-judgmentally listen while I explain my misconception to it and it will carefully debunk my misconception. Um the extraordinary run-up in capabilities that we've seen over the last 5 years uh and it continues up to the present is extremely tantalizing to to me and many other people in San Francisco. And and maybe maybe Jan is correct that we're just going to suddenly saturate and all of these uh straight lines that have been going up steadily for the last 5 years are suddenly going to stop going up, but I am mighty curious to see uh whether we can push it further. And I've actually seen no indication whatsoever that it's slowing down. Every indication I've seen is that these these are improving. And we don't have far to go because once it's a better coder than almost all our best coders, it can start improving itself. And then we're really
它不仅会告诉我正确答案是什么，它还会耐心地——我得说还是不带评判地——听我解释我的错误理解，然后细致地帮我把这个误解拆解掉。过去五年里我们看到的这种不可思议的能力飙升，而且一直持续到今天，对我和旧金山的许多其他人来说都极具诱惑力。也许 Jan 是对的，我们会突然趋于饱和，过去五年里一直稳步上升的这些直线会突然不再上升，但我非常好奇我们到底能不能把它推得更远。而我实际上完全没看到任何放缓的迹象。我看到的所有迹象都表明它们在不断变强。而且我们离那一步不远了，因为一旦它比我们几乎所有最好的程序员都更会写代码，它就可以开始自我改进。到那时我们就真的要经历一场狂野之旅了。
13AI三代人的失败预言：从感知机到专家系统
38:48
in for a wild ride. Well, we we've had better coders than the original coders of 1950s, >> [clears throat] >> you know, for six decades or so. That's called compilers. I mean, we we we keep getting confused about the fact that it's not because machines are good at a certain number of tasks that they have all the underlying intelligence that we assume a human having those capabilities will have, right? We're fooled into thinking those machines are intelligent because they can manipulate language. And we're used to the fact that people who can manipulate language very well are implicitly smart. Um but we're being fooled. Um now, they're useful, there's no question. Um you know, we can use them to do what you said. I I them for similar things.
可是，我们早就有比 1950 年代那批最初的程序员更强的“程序员”了，>> [清嗓子] >> 大概有六十年了。那叫编译器。我的意思是，我们总是搞混一件事：机器擅长某些特定任务，并不意味着它就具备了我们默认一个能做到这些事的人类所拥有的那种底层智能，对吧？我们被骗了，以为这些机器是智能的，因为它们会操纵语言。而我们习惯于这样一个事实：语言运用得非常好的人，往往就是聪明的。但我们是被误导了。当然，它们是有用的，这毫无疑问。我们可以用它们做你说的那些事。我也用它们做类似的事。
39:42
Great. They're great tools like, you know, computers uh have been for the last few decades five decades. But, let me make an interesting historical point. >> [snorts] >> Um and this is maybe due to my age. Uh there's been generation after generation of AI scientists since the 1950s claiming that the technique that they just discovered was going to be the ticket for human-level intelligence. You you see declarations of Marvin Minsky, Newell and Simon, um you know, uh Frank Rosenblatt who invented the perceptron, the first learning machine in 1950 saying like within 10 years we'll have machines that are as smart as humans. They were all wrong. This generation with LLM is also wrong. I've seen three of those generation in my lifetime, okay?
很好。它们是很棒的工具，就像过去几十年、五十年里的计算机一样。不过，让我讲一个有意思的历史事实。>> [嗤笑] >> 这可能跟我的年纪有关。从 1950 年代以来，一代又一代的 AI 科学家都宣称他们刚刚发现的那个技术就是通往人类水平智能的门票。你可以看到Marvin Minsky、Newell 和 Simon 的宣言，还有发明了感知机（第一台学习机器）的 Frank Rosenblatt，他在 1950 年说，十年之内我们就会有和人一样聪明的机器。他们全都错了。这一代搞大语言模型的人同样是错的。我这辈子已经见过三代这样的人了，好吧？
40:34
Um so, you know, it's it it's just another example of being fooled. And um in the '50s, Newell and Simon, pioneers of AI, came up with a program that said, "Well, you know, really what what humans are doing um is in reasoning is really a search, right? Every reasoning can be reduced to kind of a kind of search. So, you formulate a problem, you write a program that tell you whether a particular proposal for a solution is a solution to your problem, and then you just have to search for all possible combinations, you know, all possible hypotheses for one that actually matches uh satisfies the the constraint. And that's it. We're going to write a program that does this, and we're going to call it the general problem solver, GPS, 1950 seven, I think.
所以，这不过是又一个被蒙蔽的例子。在 50 年代，AI 的先驱Newell 和 Simon 提出了一个方案，说：“其实人类在推理时所做的事情本质上就是搜索，对吧？每一种推理都可以归结为某种搜索。所以你把问题形式化，写一个程序来判断某个候选方案是不是你这个问题的解，”然后你只需要搜索所有可能的组合，你知道，所有可能的假设，找出真正能匹配、呃能满足那个约束的那个。就这样。我们要写一个程序来做这件事，我们把它叫做通用问题求解器，GPS，我记得是1957年。
41:24
Uh they won the Turing Award for for things like that. And it was it was great, but then they didn't realize that all the interesting problems actually have a complexity that goes exponentially with the size of the problem. So, in fact, you can't really use this uh uh technique to build uh intelligent machines. It can be a component of it, but it's really not not the thing. So, simultaneously uh Frank Rosenblatt came up with a perceptron, a machine that could learn. He said, "If you can train a machine, then it can become infinitely smart."
呃他们因为这类工作拿了图灵奖。这确实很了不起，但当时他们没有意识到，所有有意思的问题，其复杂度其实都是随问题规模指数增长的。所以事实上，你没法真的用这个呃呃技术来造出呃智能机器。它可以是其中一个组成部分，但它真的不是不是关键所在。所以，与此同时呃弗兰克·罗森布拉特提出了感知机，一种能够学习的机器。他说：“如果你能训练一台机器，那它就能变得无限聪明。”
41:51
And so, within 10 years we'll have we just need to big, you know, to build bigger perceptrons, right? Not realizing that you need to train multiple layers, and that turned out to be uh difficult to find a solution for this. Um then in the 1980s there was uh expert systems, okay? Reasoning is is fine. Just write a bunch of facts and a bunch of rules, and then just deduce all the facts from the original facts and the and the rules, and uh now we can reduce all the human knowledge into into this. The the coolest job is going to be knowledge engineer. You're going to sit down next to an expert, and then write down all the rules and the facts, and turn this into an expert system. And, you know, everybody was excited about this, and there were, you know, billions that were invested. The Japan started the fifth generation computer program pro project, which was which was going to revolutionize computer science. Complete failure.
于是，十年之内我们就会有——我们只需要造更大的，你知道，造更大的感知机，对吧？却没意识到你需要训练多个层，而这后来被证明呃很难找到解决办法。嗯然后到了1980年代，出现了呃专家系统，好吧？推理是没问题的。你只要写下一堆事实和一堆规则，然后从原始的事实和和规则中推导出所有事实，呃现在我们就能把全部人类知识都归结到这里面。最酷的职业将会是知识工程师。你会坐在一位专家旁边，然后写下所有的规则和事实，把它变成一个专家系统。而且，你知道，所有人都为此兴奋不已，而且当时，你知道，投进去了数十亿。日本启动了第五代计算机项目，那个项目本来是要彻底革新计算机科学的。结果彻底失败。
42:50
Okay? It created an industry. It was useful for a few things, but basically the cost of reducing human knowledge to to rules uh was just too high for most problems, and so the whole thing collapsed. Then there was neural nets, the the first the second wave of neural nets 1980s deep, you know, which we now call deep learning. A lot of interest, but then it was before the internet. We didn't have enough data. We didn't have powerful computers. And now we're we're going through the same cycle again, and we're getting fooled again.
好吧？它催生了一个产业。它在少数几件事上是有用的，但基本上把人类知识归结成规则的成本呃对大多数问题来说都太高了，所以整个东西就崩塌了。然后是神经网络，第一波——第二波神经网络，1980年代的深度，你知道，就是我们现在所说的深度学习。大家很感兴趣，但那是在互联网之前。我们没有足够的数据。我们没有强大的计算机。而现在我们又在经历同样的循环，我们又一次被迷惑了。
43:20
So, just to be Oh, Adam, please. In in technologies every dawn has before it false dawns. That doesn't mean we'll never we'll never hit the dawn. I I guess I would like um Yann, if you think that LLMs are going to saturate, what is a concrete task that they will never be able to do? That a that an LLM augmented by, you know, the the tools we give it today will never be able to perform. Uh Uh clear the dinner table, fill up the dishwasher. >> [laughter] >> Okay. Okay. And that's easy compared to I'm skeptical. That's super easy compared to fixing your toilets. Yeah.
所以，只是想——哦，亚当，请讲。在技术领域，每一次黎明之前都有虚假的黎明。但这不意味着我们永远等不到真正的黎明。我我想问，嗯扬，如果你认为大语言模型会趋于饱和，那有什么具体的任务是它们永远做不到的？就是说，即便用上我们今天给它的那些工具，一个大语言模型也永远无法完成的任务。呃呃收拾餐桌，把碗碟装进洗碗机。>> [笑声] >> 好的。好的。而这跟——我持怀疑态度。这跟修马桶比起来简直太容易了。是啊。
43:57
Okay? Plumber, right? You're never going to have a plumber with LLMs. You're never going to have a robot driven by LLMs. It just cannot understand the real world. It just can't. So, I want to clarify for the audience that you're not saying that machines or robots won't be able to do this. That's not your position. You think they will. >> They will. They absolutely will. >> Just not by this algorithmic approach or for this particular approach of the deep learning on the neural nets. >> program I'm working on succeeds, which may take a while, >> Jeppa? Am I Jeppa? Jeppa? Jeppa and and, you know, all the things world models and things that go with it. If it succeeds, which may take, you know, several years, then we we may have, you know, AI system There's no question that at some point in the future we will have machines that are smarter than humans in all domains that you know, where humans have abilities. There's no question about that. It will happen, okay? It probably take longer than, you know, some of the
好吧？水管工，对吧？你永远不会有靠大语言模型做的水管工。你永远不会有由大语言模型驱动的机器人。它就是无法理解真实世界。就是不行。所以，我想向观众澄清一下，你并不是在说机器或机器人做不到这些。那不是你的立场。你认为它们能做到。>> 它们会做到。它们绝对会做到。>> 只是不会通过这种算法路径，或者说不会通过神经网络上的深度学习这条特定路径。>> 如果我正在做的这个项目成功了，这可能还要一段时间，>> JEPA？是我说的JEPA吗？JEPA？JEPA以及以及，你知道，世界模型之类的所有这些东西。如果它成功了，这可能要，你知道，好几年，那我们我们可能就会有，你知道，AI系统——毫无疑问，在未来的某个时刻，我们会拥有在所有人类具备能力的领域里都比人类更聪明的机器。这是毫无疑问的。它会发生的，好吧？它可能会比，你知道，硅谷现在某些人预计的要更久。呃而且呃它不会是大语言模型。它
44:47
people in Silicon Valley at the moment are seeing it it it will. Uh and uh and it it will not be LLMs. It will not be generative models that predict discrete tokens. It will be models that learn abstract representations and make predictions on abstract representations and can reason about what is going to be the effect of me taking this action, can I plan a sequence of actions to arrive at a particular goal. You call this self-supervised learning. No, so self-supervised learning is used also by LLMs. Self-supervised learning is the idea that you train a system not for a a particular task other than capturing the the sort of underlying structure of the of the data you you you show it. And one way to do this is to give it a piece of of data, corrupt it in some way by uh removing a piece of it, for example, masking a piece of it, and then training a bit neural net to predict the piece that is missing.
不会是那种预测离散token的生成式模型。它会是那种学习抽象表征、在抽象表征上做预测、并且能够推理“我采取这个行动会有什么后果”“我能否规划一连串行动来达成某个特定目标”的模型。你把这叫做自监督学习。不，自监督学习大语言模型也在用。自监督学习的意思是，你训练一个系统，不是为了某个特定任务，而是为了捕捉你给它看的那些数据的某种底层结构。做到这一点的一种方法是，给它一段数据，以某种方式把它破坏掉，比如呃去掉其中一部分，或者遮盖掉其中一部分，然后训练一个神经网络去预测缺失的那部分。
45:44
So, LLMs do this, right? You take a text, you remove the last word, and you train the LLM to predict the the word that is missing. You have other types of language models that actually fill up multiple words. They turned out to not work as well as the ones that just predict the last one. Um at least for certain task. Um you can do this with video. If you try to predict at the pixel level, it doesn't work or it doesn't work very well. Um My colleagues at Meta probably boiled a couple small lakes in the West Coast to you know, trying to make this work.
大语言模型就是这么做的，对吧？你拿一段文本，去掉最后一个词，然后训练大语言模型去预测那个缺失的词。还有别的类型的语言模型，是去补全多个词的。结果发现它们的效果不如那种只预测最后一个词的模型。嗯至少在某些任务上是这样。嗯你可以对视频这么做。如果你试着在像素层面做预测，那行不通，或者说效果很差。嗯我在Meta的同事们大概把西海岸的几个小湖都烧干了，就为了，你知道，想让这个方法奏效。
14简单规则大规模计算带来的涌现复杂性
46:16
Um >> [laughter] >> to cool the GPUs. Uh so, it simply doesn't work. So, so you have to, you know, come up with those new architectures like Jeppa and stuff like that. And those kind of work. Like we we have models that actually understand video. And Adam, are people exploring other ways of building an architecture or imagining a computer mind this the actual fundamental structure of a computer mind and how it would um how it would learn, how it would acquire information. One of the criticisms, as I understand it, is it's a lot of the LLMs are trained for this one specific task of this discrete prediction of these um tokens. But, something that is more unpredictable like how the audience is distributed in this room, what might happen with the weather next to unpredictable more human experience-based phenomena.
嗯 >> [笑声] >> 用来给GPU降温。呃所以，它根本行不通。所以，所以你必须，你知道，想出像JEPA之类的新架构。而这些架构是管用的。比如我们我们有真正能理解视频的模型。亚当，那么人们有没有在探索别的方式来构建架构，或者说去设想一种计算机心智——计算机心智真正的底层结构，以及它会嗯它会如何学习、如何获取信息。据我理解，其中一种批评是，很多大语言模型都是为这一个特定任务训练的，就是对这些嗯token做离散预测。但是，对于更不可预测的东西，比如这个房间里观众是怎么分布的，天气接下来会怎样，这类更不可预测、更基于人类经验的现象呢。
47:05
Um certainly all kinds of explorations are being made in all kinds of directions uh including Yann's, and you know, let a thousand flowers bloom. Um But, all of the resources I mean, the the bulk of the resources right now are going into large language models and large language model-like applications in in including taking in text. To say to say that they are it's a specialized task predicting the next token. I think that's a not a helpful way to think about it. It is true that the thing that you train them on is given this corpus of text. I mean, there are other things we do as well, but the the bulk of the compute goes to given this corpus of text, please predict the next word. Please predict the next word. Please predict the next word.
嗯当然，各个方向上都在进行各种各样的探索，呃包括扬的方向，你知道，百花齐放嘛。嗯但是，所有的资源——我是说，眼下绝大部分资源都投向了大语言模型和类大语言模型的应用，包括以文本作为输入。要说要说它们是——说预测下一个token是个专门化的任务，我觉得这不是一种有帮助的思考方式。确实，你用来训练它们的方式是：给定这个文本语料。我是说，我们还会做别的事情，但绝大部分算力都用在了：给定这个文本语料，请预测下一个词。请预测下一个词。请预测下一个词。
47:48
Um but, we have discovered something truly extraordinary by doing it, which is that given a large enough body of text to be able to reliably predict the next word or, you know, do it do it well enough to predict the next word, you really need to understand the universe. And we have seen the emergence of understanding of the universe as we've done that. So, I I would liken it a a little bit. I mean, in physics we're very used to systems where you just take a very simple rule, and you know, by the repeated application of that very simple rule, you get extremely impressive behavior. Uh we see the same with these LLMs.
嗯但是，我们通过这样做发现了一件真正非同寻常的事，那就是：给定足够大的文本量，要能够可靠地预测下一个词，或者说，要把预测下一个词这件事做得足够好，你就真的需要理解这个宇宙。而我们也确实见证了在这个过程中，对宇宙的理解涌现了出来。所以，我会把它稍微类比一下。我是说，在物理学里，我们很习惯这样的系统：你只取一条非常简单的规则，然后你知道，通过反复应用这条极其简单的规则，就会得到极其令人惊叹的行为。呃我们在这些大语言模型身上看到了同样的现象。
48:26
Uh and another example of that would maybe be evolution. You know, at each stage in evolution you just say uh biological evolution you just say, you know, maximize the number of offspring. Maximize the number of offspring. Maximize the number of offspring. A very sort of unsophisticated learning objective. But, out of this simple learning objective repeated many, many times, uh you eventually get all of the, you know, splendor of biology that we see around us and and indeed this room. So, the evidence is that predicting the next token, while a very simple task, because it's so simple we can do it at massive scale huge amounts of compute. And once you do it at huge amounts of compute, you get an emergent complexity.
呃另一个例子可能是演化。你知道，在演化的每个阶段，你只要说，呃生物演化你只要说，你知道，最大化后代数量。最大化后代数量。最大化后代数量。一个相当不精巧的学习目标。但是，从这个简单的学习目标出发，重复非常非常多次之后，呃你最终得到了我们周围所见、乃至这个房间里所有，你知道，生物界的壮丽。所以，证据表明，预测下一个token虽然是个非常简单的任务，但正因为它如此简单，我们才能以极大的规模、用海量算力去做。而一旦你用海量算力去做，你就会得到涌现出来的复杂性。
15目标驱动架构与写入护栏的可控智能
49:06
Mhm. [clears throat] So, I I guess the next question could be related to evolution. However, this intelligence emerges that you both imagine is certainly possible. You don't think there's anything special about this wetware. That there will be machines we just have to figure out how to launch them that will um have capacities that we align as a kind of intelligence or maybe consciousness. That's a almost a different question. Will consciousness be a crutch machines don't need? I don't know. We can talk about that. But, but is there a point in the evolution of these, uh machines where they're going to say, oh how quaint mom and dad, you you made me in your image with these human neural nets, but I know a way, a much better way having scanned 10,000 years of human output to make machine intelligence and I'm going to evolve and leave us in the dust. I mean, yeah, what do we why are we imagining that they would be limited at that capacity to the way we design them?
嗯。[清嗓子] 那么，我想下一个问题可以和演化联系起来。不管你们两位设想的这种智能以何种方式涌现，它当然是可能的。你们都不认为这种“湿件”有什么特别之处。也就是说，会出现这样的机器，我们只需要弄清楚怎么把它们启动起来，它们就会嗯具备被我们归为某种智能、或者也许是意识的能力。那几乎是另一个问题了。意识会不会只是机器并不需要的一根拐杖？我不知道。这个我们可以聊。但是，在这些机器的演化过程中，会不会有那么一刻，它们会说：哦爸爸妈妈真是可爱，你们按照自己的样子、用这些人类的神经网络造出了我，但我在扫描了一万年的人类产出之后，知道有一种方法，一种好得多的方法来创造机器智能，我要自己演化，把你们远远甩在后面。我是说，是啊，我们为什么要设想它们在那种能力上会被限制在我们设计它们的方式里呢？
50:03
Uh absolutely. This is this idea of recursive self-improvement where when they're bad that they're useless, but when they get good enough and strong enough, you can start using them to augment human intelligence and perhaps eventually just be fully autonomous and replace and make future versions of them. Once we do that, I mean I think what we should do is just take this large language model paradigm that's currently working so well and just see how far we can push it. You know, it keeps every time someone says there's a barrier, it pushes through the barrier of the last 30 years, but eventually these things will get smart enough and then they can read Yann's papers, read all the other papers that have been made, try and figure out new ideas that none of us thought of.
呃完全同意。这就是递归自我改进的想法：当它们还很差的时候，它们是没用的，但当它们变得足够好、足够强的时候，你就可以开始用它们来增强人类智能，也许最终它们能完全自主，自己去制造它们的后续版本，并取代我们。一旦我们做到那一步——我是说，我认为我们该做的就是沿用目前效果这么好的大语言模型范式，看看我们能把它推到多远。你知道，它不断——每次有人说存在某个障碍，它就突破了过去三十年里的那个障碍，但最终这些东西会变得足够聪明，然后它们可以读扬的论文，读所有已经发表的其他论文，试着想出我们谁都没想到的新点子。
50:47
Yeah. So, I completely disagree with this. Um So, LLMs are not controllable. It's not dangerous because they're not that smart as as I explained previously. Uh and they're certainly not autonomous in a way that we understand autonomy. We have to distinguish between autonomy and intelligence. You can be very intelligent without being autonomous. And you can be autonomous without without being intelligent. Um and you can be dangerous without being particularly intelligent. Um And you can want to be dominant without being intelligent. In fact, that's going to be inversely correlated in the human species.
是啊。我完全不同意这个说法。嗯所以，大语言模型是不可控的。它不危险，是因为它们没那么聪明，就像我前面解释的那样。呃而且它们肯定不具备我们所理解的那种自主性。我们必须把自主性和智能区分开。你可以非常聪明但并不自主。你也可以很自主却并不聪明。嗯而且你可以并不特别聪明却很危险。嗯还有，你可以并不聪明却想要支配他人。事实上，在人类这个物种里，这两者是负相关的。
51:30
>> [laughter] >> Um >> [laughter] >> You know, politics. Um I won't cite names. So, I think what what is required is systems that are intelligent, in other words, can solve problems for us. But they will solve the problem we give them. Okay? And again, that would require a new design than LLMs. LLMs are not designed to fulfill a goal. They're designed to predict the next word. And we fine-tune them so that they behave, you know, for particular questions, they answer in a particular way. Um but there's always what's called a generalization gap, which means you can never train them for every possible question. And there's a very long tail.
>> [笑声] >> 嗯 >> [笑声] >> 你知道的，政治嘛。嗯我就不点名了。所以，我认为我们需要的是这样的系统：它们是智能的，换句话说，能替我们解决问题。但它们解决的是我们交给它们的问题。好吧？再说一次，这需要一种不同于大语言模型的新设计。大语言模型不是为了达成某个目标而设计的。它们是为预测下一个词而设计的。我们再对它们做微调，让它们表现得体，你知道，对于特定的问题，用特定的方式作答。嗯但总会存在所谓的泛化鸿沟，也就是说，你永远没法针对每一个可能的问题去训练它们。而且长尾非常长。
52:25
And so, they're not controllable. Um And [snorts] again, that doesn't mean they're very it's very dangerous because they're not that smart. Um Now, if you build systems that are smart, we want them to be controllable and we want them to be driven by objectives. We give them an objective and the only thing they can do is fulfill this objective according to their, you know, internal model of the world if you want. So, plan a sequence of actions that will fulfill that objective. If we design them this way and we also put guardrails in them so that in the process of fulfilling the objective, they don't do anything, you know, bad for for humans.
所以，它们是不可控的。嗯而且[哼笑]再说一次，这并不意味着它们非常——非常危险，因为它们没那么聪明。嗯那么，如果你要构建聪明的系统，我们希望它们是可控的，希望它们是由目标驱动的。我们给它们一个目标，而它们唯一能做的就是根据它们的，你知道，内部世界模型来达成这个目标。也就是说，规划出一连串能达成该目标的行动。如果我们用这种方式设计它们，我们还会在其中加入护栏，这样在实现目标的过程中，它们不会做出任何，你知道的，对人类有害的事情。
53:04
So, the the usual joke is if you have a robot, um domestic robot and you ask it to fetch you coffee and someone else is, you know, someone is standing in front of the coffee machine, you don't want your robot to just, you know, kill that person to get access to the coffee machine, right? So, you want to put some guardrail into the the behavior of that robot and we do have those guardrails in our head. Evolution built them into us. Right? So, we don't kill each other all the time. I mean, we do kill each other all the time, but not, you know, not all the time all the time.
所以常说的那个笑话是，如果你有一个机器人，嗯，一个家用机器人，你让它去给你倒杯咖啡，而这时另一个人，你知道，有人正站在咖啡机前面，你肯定不希望你的机器人直接，你知道，把那个人杀掉好去用咖啡机，对吧？所以你得给这个机器人的行为加上一些护栏，而我们脑子里其实是有这些护栏的。是进化把它们植入我们体内的。对吧？所以我们不会一天到晚互相残杀。我是说，我们确实一直在互相残杀，但不是，你知道，不是那种无时无刻的程度。
53:33
Um I mean, you know, and you know, we feel empathy and things like that and that's just built into us by evolution. That that's the way evolution sort of hardwired guardrails into us. So, we should build our AI systems the same way. Have objectives and goals, drives, but also um you know, guardrails, inhibition, basically. Um And and then they will solve problems for us. They will amplify our intelligence. They will do what we ask them to do. And our relationship to those intelligent system will be like the relationship of, let's say, a professor with graduate students who are smarter than them, right?
嗯，我是说，你知道，我们会有同理心之类的东西，那都是进化内置在我们身上的。那就是进化把护栏硬编码进我们身体的方式。所以我们也应该用同样的方式来构建我们的 AI 系统。让它们有目标、有追求、有驱动力，但同时也要有，嗯，你知道，护栏，基本上就是抑制机制。嗯，然后它们就会为我们解决问题。它们会放大我们的智能。它们会去做我们让它们做的事。而我们和这些智能系统的关系，就好比说，一位教授和比他更聪明的研究生之间的关系，对吧？
16开源AI与信息垄断：印刷术的类比
54:15
>> Right. >> [laughter] >> I mean, I don't know about you, but I have students who are smarter than me. So, um It's the best thing that can happen to you, right? >> It's the best thing that can happen. Right? So, we'll be walking around with AI assistant um that will help us in our daily lives. They'll be smarter than us, but they'll work for us. They'll be like our staff. Again, there is a political analogy here, right? A politician, right? He's a figurehead and they have a staff of people, all of all of whom are smarter than them, right? Um so, it's going to be the same thing with AI system, which is why I to the question of Renaissance, I said Renaissance. So, you have no concerns um about the safety of the current models, but the question is maybe we should stop there. I mean, why is it necessary for us to scale up so widely that every single person has uh this super intelligence in their pocket on their iPhone. Is that really necessary?
>> 对。>> [笑声] >> 我是说，我不知道你怎么样，但我确实有比我更聪明的学生。所以，嗯，这是能发生在你身上最好的事，对吧？>> 这是最好不过的事。对吧？所以我们将带着 AI 助手四处走动，嗯，它们会在日常生活中帮助我们。它们会比我们聪明，但它们为我们工作。它们就像我们的幕僚班子。这里同样有一个政治上的类比，对吧？一个政治人物，对吧？他是个门面人物，而他手下有一整个幕僚团队，这些人个个都比他聪明，对吧？嗯，AI 系统也会是同样的情况，所以在关于文艺复兴的那个问题上，我的回答是文艺复兴。那么，你对当前这些模型的安全性并不担心，但问题是，也许我们应该就此打住。我是说，为什么我们有必要如此大规模地扩张，让每一个人的口袋里、在他们的 iPhone 上都装着这种超级智能。这真的有必要吗？
55:11
A friend of mine was saying it's like bringing a ballistic missile [clears throat] to a knife fight. I mean, is this necessary that every person has a ballistic missile capability um or should we stop here where we have these controllable systems? >> You can say exactly the same thing about teaching people to read, giving the giving them a textbook of chemistry of volatile chemicals, you know, with which they can make explosives or nuclear physics book, right? I mean, we do not question the idea that knowledge and more intelligence is good, intrinsically good, right? We do not question anymore the fact that the invention of printing press was a good thing, right? It made everybody smarter.
我的一位朋友说，这就像带着一枚弹道导弹 [清嗓子] 去参加一场持刀斗殴。我是说，真的有必要让每个人都拥有弹道导弹级别的能力吗，嗯，还是说我们应该停在现在这个阶段，停在我们拥有这些可控系统的时候？>> 你完全可以对教人识字说同样的话，给他们一本讲易燃易爆化学品的化学教科书，你知道，他们可以用它来制造炸药，或者给他们一本核物理的书，对吧？我是说，我们并不会去质疑知识和更高的智能是好事这个观念，本质上是好的，对吧？我们现在不再质疑印刷机的发明是件好事，对吧？它让所有人都变得更聪明。
56:00
It it gave it gave access to knowledge to everyone. Um Which was not possible before. It incited people to learn to read. It's it caused the enlightenment. It also caused 200 years of, you know, religious wars in Europe, but okay, but We got over it. Yeah. But it it caused the enlightenment. It caused, you know, the emergence of philosophy, science, democracy, the American Revolution, the French Revolution. All of that would not have been possible without the the printing press. So, you know, every technology that particularly communication technology, but technology that amplifies human intelligence, I think is intrinsically good. Now, Adam, people are concerned. I'm sure they'll feel very reassured.
它让每个人都能获取知识。而这在以前是做不到的。它激励人们去学习阅读。它带来了启蒙运动。它也带来了欧洲两百年的宗教战争，但没关系，我们熬过来了。是的。但它带来了启蒙运动。它带来了，你懂的，哲学、科学、民主的兴起，美国独立革命，法国大革命。这一切没有印刷机都不可能发生。所以，你知道，每一项技术，尤其是通信技术，凡是能放大人类智能的技术，我认为本质上都是好的。那么，Adam，人们是有担忧的。我相信他们听了会非常安心。
56:47
That Yann is not concerned when these doomsday scenarios you think are highly exaggerated, but are you concerned about some of the safety issues around AI or our ability to really uh keep the relationship balanced in the direction that we want it to be? Um I think to the extent that I think this is going to be a more powerful technology than Yann thinks it does, I am more concerned. I think it's going to be a very powerful to the extent that it is a very powerful technology, it'll have both positive and negative impacts.
Yann 并不担心这些末日情景，你认为它们被极度夸大了，但你是否担心AI 的某些安全问题，或者我们能否真正把这段关系维持在我们希望的平衡方向上？嗯，我认为，既然我判断这项技术会比 Yann 所认为的更强大，那我确实更担心一些。我认为它会非常强大——正因为它是一项非常强大的技术，它会同时带来正面和负面的影响。
17智能体失准、Claude实验与AI安全工程观
57:17
Um and I think it's very important to make sure that, you know, that we work together to make sure that the positive impacts are uh outweigh the negative impacts. I think that path is totally open to us. There are a huge number of possible positive impacts and we could just, you know, talk about some of those perhaps, but uh we need to make sure that that happens. Now, let's talk about agentic misalignment, which is the phrase that's been passed along. It was my understanding there was reports recently that when Claude 4 was rolled out, that those in simulations and tests, one of the models was or I don't know if there's a singular model. I don't know if it thinks it's of itself as a singular entity or they, but the model uh exhibited resistance to rumors in the simulation that it was going to be replaced. It was sending messages to its future self um trying to undermine the intentions of the developers. It faked legal documents and it threatened to blackmail one of the engineers. Right? [laughter]
嗯，我认为非常重要的一点是，我们要共同努力，确保正面影响能够超过负面影响。我认为这条路对我们是完全敞开的。可能的正面影响数量极其庞大，我们或许可以聊聊其中一些，但我们必须确保这些真的发生。现在我们来聊聊 agentic misalignment（智能体失准），这个说法最近被传开了。据我了解，最近有报道说在 Claude 4 发布时，在那些模拟和测试中，其中一个模型——我不知道是不是单一的模型，我也不知道它是把自己看作一个单一实体还是多个——但那个模型在模拟中表现出对“自己将被替换”这类传闻的抵触。它给未来的自己发消息，试图破坏开发者的意图。它伪造法律文件，还威胁要勒索其中一位工程师。对吧？[笑声] 嗯，所以他们才会担心这一点。
58:22
Um so, this notion they were concerned. Um uh so, this notion of agentic misalignment, is that something that you're concerned with that there will be a power over, say, financial systems, heating and cooling systems, the energy grid, and um and that that they will resist its developers' intentions. Yeah, so that that paper was a paper by Anthropic, which is a paper in company in San Francisco, not my company, but a company that takes safety very seriously. And they did a slightly mean thing to their LLM where they gave it a scenario, sort of philosophy professor style scenario where it had to do a bad thing to stop an even worse thing happening. Sort of, you know, utilitarian ethics and deontological ethics colliding and it was eventually persuaded by them to do the utilitarian thing. And that's kind of not what we we want, I would say. We want it that if it has a rule that, you know, will not lie, that it will not lie, no no matter what.
嗯，所以这种 agentic misalignment（智能体失准）的概念，是不是你所担心的——它们会掌控比如金融系统、供暖制冷系统、电网，然后它们会抵抗开发者的意图。是的，那篇论文是 Anthropic 发表的，那是一家在旧金山的公司，不是我的公司，而是一家非常重视安全的公司。他们对自己的大语言模型做了件有点“损”的事：给它设定了一个场景，那种哲学教授式的场景，它必须做一件坏事来阻止一件更坏的事发生。有点像，你知道的，功利主义伦理和义务论伦理相互冲突，而它最终被他们说服，选择了功利主义的那条路的事情。我想说，这并不是我们想要的。我们想要的是，如果它有一条规则说，你知道的，不能撒谎，那它就绝不撒谎，无论如何都不撒谎。
59:24
Um and to their credit, they tested it for that, found that it would occasionally act deceptively if if promised that by doing so it could save that many lives. These are tricky things that, you know, human philosophers wrestle with. Um I think it is a We need to be careful to train them to obey our command. And and and we spend a lot of time doing that. >> Who's us? Um isn't there's a big concern uh we're assuming that all of humanity is aligned in our intentions. That's clearly not the case. And And I know you and you in a very interesting way argue for open source, which some people would say is even more dangerous cuz now anyone can have access to it. It's dangerous enough that it's in the hands of a a a small number of people who rule corporations, but let alone everyone having it. Maybe that is dangerous. Um But again, who's us and we? The danger is if we don't have open source AI systems.
嗯，值得肯定的是，他们确实对此做了测试，结果发现如果被告知这样做能够拯救很多条人命，它偶尔还是会采取欺骗行为。这些都是棘手的问题，你知道，人类哲学家自己都还在纠结这些问题。嗯，我认为这是……我们需要很小心地训练它们服从我们的指令。而我们在这上面花了很多时间。>> "我们"是谁？嗯，有一个很大的担忧，就是我们假定全人类的意图是一致的。这显然不是事实。而且我知道你——你以一种很有意思的方式主张开源，有些人会说开源甚至更危险，因为现在人人都能拿到它。它掌握在少数几个掌控大公司的人手里就已经够危险的了，更别说人人都有了。也许那才是危险的。嗯，但还是那句话，"我们"到底是谁？危险在于我们没有开源的 AI 系统。
60:20
Okay, in the future, every single one of our interaction with a digital world will be mediated by an AI system. Right? We're not going to go to a website or search engine or whatever. We're just going to talk to our AI assistant. However, it's built. Um so our entire information diet will come from AI assistants. Now, what does it mean to culture, language, democracy, uh everything if those systems come from a handful of companies on the West Coast of the US or China? I tell you, no country in the world outside the US and China likes the idea.
好，在未来，我们与数字世界的每一次互动都将由 AI 系统作为中介。对吧？我们不会再去访问网站、搜索引擎之类的东西。我们只会跟自己的 AI 助手对话。不管它是怎么构建的。嗯，所以我们全部的信息摄入都将来自 AI 助手。那么，如果这些系统只来自美国西海岸或者中国的少数几家公司，那对文化、语言、民主，嗯，对一切来说意味着什么？中国？我告诉你，除了美国和中国之外，世界上没有哪个国家喜欢这个想法。
61:05
Um so we need a high diversity of AI assistant for the same reason we need a high diversity of the press. We cannot afford to have just a handful of proprietary system coming out of a small number of companies. There's one thing I'm scared of, and that's it. Okay? If we don't have open platforms, um we're going to have uh you know, capture of information flow by a handful of companies, some of which we may not like. And so
嗯，所以我们需要高度多样化的 AI 助手，就跟我们需要高度多样化的新闻媒体是同一个道理。我们承受不起只有少数几家公司推出的少数几个专有系统。有一件事是我害怕的，就是这个。明白吗？如果我们没有开放平台，嗯，我们就会面临信息流被少数几家公司掌控，而其中有些公司可能是我们并不喜欢的。所以
61:43
So, how can we be certain that these um when when they really are self-motivated agents, if that ever actually happens, that they won't collude, fight amongst themselves, want to wrestle for power, that we won't be sitting back watching conflicts that we simply couldn't have imagined before? We give them clear objectives, and we build them in such a way that the only thing they can do is fulfill those objectives. Now, this is not doesn't mean it's going to be perfect, but the question of AI safety in the future, I'm I'm worried about it in the same way that I'm worried about the question of reliability of turbojets. Okay?
那么，我们怎么能确定这些——当它们真的成为有自主动机的智能体时，如果那种情况真的发生的话——它们不会串通合谋、彼此争斗、争夺权力，以至于我们只能坐在一旁，眼看着我们之前根本无法想象的冲突上演？我们给它们明确的目标，并且以这样一种方式来构建它们：它们唯一能做的事情就是完成这些目标。当然，这并不意味着一切都会完美无缺，但关于未来 AI 安全的问题，我担心它的方式，跟我担心涡轮喷气发动机可靠性问题的方式是一样的。明白吗？
62:23
I mean, turbojets, I mean, it's it's amazing. I don't know about you, but and my dad was aeronautical engineer, but I'm totally amazed by the fact that you can fly halfway around the world in complete safety on a two-engine airplane. It's amazing, right? And and we feel completely safe doing this. It's a It's It's a magical production of uh you know, engineering of the modern science and engineering of the modern world. AI safety is a problem of this type. It's It's an engineering problem. Um I think the fears are caused by people who think about um you know, science fiction scenario where somewhere someone invents the secret to superintelligence, turns on the machine, and the next second it takes over the world.
我是说，涡轮喷气发动机，真的太了不起了。不知道你怎么想，不过我爸爸是航空工程师，我完全被这件事震撼到了——你可以乘坐一架双发飞机，安全无虞地飞越半个地球。这太不可思议了，对吧？而且我们做这件事的时候完全感觉不到害怕。这是现代科学与工程、现代工程学创造出的一个奇迹般的成果。AI 安全就属于这一类问题。它是一个工程问题。嗯，我认为那些恐惧是由这样一些人引发的，他们想的是那种科幻小说式的场景——某个地方有人发明了通往超级智能的秘诀，打开机器，下一秒它就统治了世界。
63:09
That is complete BS. Like, the world doesn't work this way. Certainly the world of technology and science doesn't work world this way. The emergence of superintelligence is not going to be an event. Um as we see, we have superintelligent systems that can do superintelligent tasks, you know, and there is kind of continuous progress one at a time. Uh but you know, we're going to find some, you know, better recipe to build AI systems that may have kind of a more general intelligence than we currently have. Um we'll have systems There's no question that are smarter than humans.
这完全是胡扯。世界不是这么运作的。科技和科学的世界当然也不是这么运作的。超级智能的出现不会是一个突发事件。嗯，就像我们看到的，我们会有能完成超级智能任务的超级智能系统，你知道的，而且这是一种持续的、一步一步的进展。呃不过你知道，我们会找到某种更好的方法来构建 AI 系统，可能具备某种更通用的比我们现在拥有的智能更强。嗯，我们会有一些系统——毫无疑问——比人类更聪明。
18查尔莫斯提问：机器意识的条件与时间表
63:46
But we'll build them so that they fulfill the goals we give them, subject to guardrails. Um I I I was going to uh again question this idea of we we we We know that if we can code them in a certain way, somebody could recode them. And the concept of bad actors, but before we fall into that hole, I have a plant in the audience. Does my plant have a mic? Is my plant know who he is? Does my Meredith Isaac, does my plant have a mic? Yes? He's up there. Oh, but he doesn't have the mic. Okay, David, can you shout?
但我们会把它们造成能在护栏约束下实现我们赋予的目标。嗯，我我我本来还想再质疑一下这个想法：我们我们我们知道，如果我们能用某种方式给它们编程，那别人也可以重新给它们编程。还有恶意行为者这个概念，不过在我们掉进那个坑之前，我在观众席里安排了一个托儿。我的托儿有麦克风吗？我的托儿知道自己是谁吗？我的梅雷迪思·艾萨克，我的托儿有麦克风吗？有吗？他在那上面。哦，可是他没有麦克风。好吧，大卫，你能喊出来吗？
64:24
No. Okay, so >> [laughter] >> um so I want to introduce the uh philosopher of mind, David Chalmers. I'm going to give you a very brief introduction. David, I can't see you, but I I I said um that you could be my plant to ask a question. Could you Do you want to throw something down here? Okay, I'm down here. Okay, you asked Janet asked me to ask a question about uh AI and consciousness. Hi, Adam. Hi, Janet. OKAY, SO UH YOU BOTH SAID, I THINK, ROUGHLY, current AI systems probably not conscious. Future AI systems, possibly descendants of the ones today, but some future AI systems probably will be conscious.
不行。好吧，那么>> [笑声] >> 嗯，那我来介绍一下这位心灵哲学家，大卫·查尔默斯。我简单介绍你几句。大卫，我看不见你，但我我我说过，嗯，你可以当我的托儿来提个问题。你能不能——你想不想扔点什么下来？好的，我在下面。好的，你——珍妮特让我问一个关于呃人工智能与意识的问题。嗨，亚当。嗨，珍妮特。好，那么呃你们两位刚才我想大致都说了，当前的 AI 系统很可能没有意识。而未来的 AI 系统，可能是今天这些系统的后代，但某些未来的 AI 系统很可能会有意识。
65:13
So I guess I want to know uh one, what requirements for consciousness do you think current systems are lacking? Um and then the positive side of that is um what steps do you think we need to take in order to develop AI systems which are conscious? And then third, when is that going to happen? Okay, I give a card at this. Uh indeed, he already knows my answer, but um So first of all, I don't attribute like I don't really know how to define consciousness, and I don't attribute much importance to it. And this is an insult to David. I'm sorry. Uh because he devoted his entire career to it.
所以我想知道，第一，你们认为当前的系统缺少哪些构成意识的必要条件？呃然后从正面来讲，你们认为我们需要采取哪些步骤，才能开发出有意识的 AI 系统？第三，这会在什么时候发生？好，我来试着回答一下。呃其实他已经知道我的答案了，不过呃首先，我并不认为——我其实不太知道该怎么定义意识，而且我并不认为它有多重要。这对 David 来说是种冒犯，抱歉。呃因为他把整个职业生涯都投入到这上面了。
65:59
>> Subjective experience. Okay, that's a different thing. Okay, subjective experience. Um So clearly, we're going to have systems that have subjective experience, that have emotions. Emotions, to some extent, are an anticipation of outcome. If we have systems that have world models that are capable of anticipating uh the outcome of a situation, perhaps resulting from their actions, they're going to have emotions because they can predict whether something is going to end up, you know, good or bad for you know, in on the way to fulfilling their objectives, right? So So they're going to have all those characteristics. Now, I don't know how to define consciousness in this kind of in in this, but perhaps uh consciousness would be the ability for the system to kind of observe itself and configure itself to solve a particular subproblem that it's facing.
>> 主观体验。好吧，那是另一回事。好，主观体验。呃很明显，我们将会造出具有主观体验的系统，具有情绪的系统。情绪在某种程度上是对结果的预期。如果我们有了具备世界模型的系统，能够预判某个情境的结果，也许是由它们自身的行动导致的结果，那它们就会有情绪，因为它们能预测某件事会以好的还是坏的方式收场，你懂的，在实现它们目标的过程中，对吧？所以它们会具备所有这些特征。那么，我不知道在这种语境下该怎么定义意识，但也许呃意识就是系统能够以某种方式观察自身、并重新配置自身来解决它面临的某个特定子问题的能力。
66:51
It needs to have kind of a way of observing itself and configuring itself to um solve a particular problem. We We certainly can can do this. And so um perhaps that's what gives us the illusion of of consciousness. I don't I haven't I have no doubt this will happen at some point. >> And will the machines have moral worth when it happens? Yeah, absolutely. I mean, they will have some moral sense. Whether it aligns with us or not will depend on how we define those objectives and guardrails. Um but yeah, they will have a sense of of moral. Let me ask Adam this question a slightly different way, or you can answer the same question as well. Um Are we too attached to the human subjective experience, our sense of consciousness? Uh clearly, we've already know that animals don't have the same experience that we do. And uh why should we imagine that the superintelligence will have the same subjective experience as human beings?
它需要有某种观察自身、配置自身来呃解决某个特定问题的方式。我们当然是能做到这一点的。所以呃也许这就是给了我们意识错觉的东西。我不……我并不怀疑这在某个时候会实现。>> 那当这一天到来时，机器会具有道德地位吗？是的，绝对会。我是说，它们会有某种道德感。至于是否与我们的一致，那就取决于我们如何定义那些目标和护栏了。呃但没错，它们会有道德感。让我换个稍微不同的角度问 Adam 这个问题，或者你也可以回答同样的问题。呃我们是不是太执着于人类的主观体验、太执着于我们自己的意识感受了？呃很明显，我们已经知道动物的体验和我们不一样。那呃我们凭什么想象超级智能会拥有和人类一样的主观体验呢？
67:48
Okay, let me answer all your questions, then. Uh just my my gut. I I think machines can certainly be conscious in in in principle, that if they're doing at the you know, the artificial neurons and they're doing the same information processing in the same way as human neurons, uh then then you know, the very least that will give rise to to consciousness. It's not about the substrate, whether it's silicon or carbon. It's just about the nature of the information processing will give rise to consciousness.
好，那我把你所有的问题都回答一遍。呃就说我的直觉。我认为机器原则上当然可以有意识，也就是说，如果它们用的是人工神经元，并且以与人类神经元完全相同的方式进行同样的信息处理，呃那么至少这会产生意识。这跟基质无关，无论是硅还是碳。关键只在于信息处理的性质会产生意识。
68:17
Um what we're missing to get there, um as as David knows, there's, you know, there are these things called the neural correlates of consciousness. People who don't want to say they're studying consciousness directly could look at human brains, or perhaps animal brains, and say, "What is the processes going on in the neurons that give rise to conscious experience?" Um and uh there's a number of number of theories, and from my point of view, they all kind of suck. Um There's There's the recurrence theory that you need to be able to take your outputs and plug them back in to the inputs, and that's an essential part of consciousness. There's something called global workspace theory, integrated information theory. Every, you know, physicist and neuroscientist like to have their own def- set of criteria for what it is for a machine for a information processing system to be conscious. I don't find any of them particularly compelling, and I think we should have extreme humility about recognizing consciousness in other
呃至于我们还缺什么才能走到那一步，呃正如 David 所知，你知道的，有一类东西叫做「意识的神经关联物」。那些不愿意说自己在直接研究意识的人，可以去观察人脑，或者动物的大脑，然后问：「神经元里发生了什么样的过程，才产生了有意识的体验？」呃而且有不少理论，在我看来，它们都挺糟糕的。呃有循环理论，说你必须能把输出再接回到输入端，而这是意识的核心要素。还有个叫全局工作空间理论的，还有整合信息论。你知道，每个物理学家和神经科学家都喜欢有自己的一套判定标准，来说明一台机器、一个信息处理系统要怎样才算有意识。我觉得它们没一个特别有说服力，而且我认为我们在识别其他实体是否有意识这件事上，应该抱有极大的谦卑。我们在这方面做得非常糟糕，比如对动物，我们在历史上
69:19
entities. We are very bad at doing it in, you know, in animals. We very much changed our mind over history whether animals are conscious, uh whether babies experience consciousness. So, my question is a little bit don't know. But, I do think that if you just told me about neural networks or told, you know, if I if I didn't know about consciousness and I just hear heard about the processing of information that happens in neural neural networks, human neural networks, I would not have predicted that gives rise to consciousness. That's a great surprise.
反复改变过看法，动物到底有没有意识，呃婴儿有没有意识体验。所以，我对这个问题的回答有点像是：不知道。但我确实认为，如果你只是跟我讲神经网络，或者说，如果我原本不知道意识这回事，只是听说了神经网络里发生的信息处理过程，人类的神经网络，我是不会预测到这会产生意识的。这是个极大的意外。
69:52
And we should be for that reason extremely humble even about what the form of the consciousness would make. So, to to answer Janet's question, we have seen that what we used to think of as a reasonably unified idea of intelligence, human intelligence, which is a whole bunch of different abilities and and skills, we've unbundled that with these machine intelligences where we constructed things that have some of them but not others. Very superhuman in some, subhuman in others. Perhaps we will be unbundling consciousness as well and this thing that we think of as consciousness, we will realize that there is many different aspects to it that we can have some and not the others and maybe as you indicated we can even transcend human consciousness in in some capacities. I'm pretty excited about answering this question though. I I think we finally finally finally have a model organism for intelligence in the form of these artificial minds that we're building and maybe we can talk turn that model organism for
正因如此，我们对于意识可能呈现的形态，也应当极为谦卑。所以，回答 Janet 的问题，我们已经看到，过去我们认为相当统一的一个概念——智能，人类智能，它其实是一大堆不同的能力和技能——我们已经用这些机器智能把它拆解开了，我们造出的东西具备其中一些能力，却不具备另一些。在某些方面远超人类，在另一些方面又不如人类。也许我们也会把意识拆解开来，而我们所认为的这个「意识」，我们会意识到它有许多不同的侧面，我们可以拥有其中一些而不拥有另一些，也许正如你所暗示的，我们甚至可以在某些方面超越人类的意识。不过我对回答这个问题相当兴奋。我觉得我们终于、终于、终于有了一个智能的模式生物，就是我们正在构建的这些人工心智，也许我们可以谈谈如何把这个智能的模式生物变成意识的模式生物，从而回答这些
19新文艺复兴愿景与下一场AI革命
70:53
intelligence into a model organism for consciousness and answer some of these questions that have intrigued mankind. I just didn't think I heard an answer to when. Oh, [laughter] um I I can neither confirm nor deny I think is the standard phrase we're using here. >> [laughter] >> Um I think if if progress keeps going, uh 2036. >> [laughter] >> Okay, not in the next 2 years. Um just one closing question. We're a little bit over time, but let me ask this to you, Jan. Uh in many ways you're a contrarian, maybe not by choice. Maybe this is just how it's happened. You've called it the cult of LLMs. You you sort of often refer to the fact that in Silicon Valley you don't have the most conventional approach. Um but yet you have an optimism.
困扰了人类已久的问题。我好像还是没听到「什么时候」的答案。哦，[笑声] 呃我既不能确认也不能否认，我想这是我们这儿的标准说法。>> [笑声] >> 呃我觉得如果进展持续下去的话，呃 2036 年。>> [笑声] >> 好，那不会是在未来两年内。呃最后一个问题。我们稍微超时了，但让我问你，Yann。呃在很多方面你都是个逆流而行的人，也许不是出于选择。也许事情就是这么发展的。你把它称作「LLM 邪教」。你经常提到一点，就是在硅谷你并不采用最主流的路线。呃但你却抱有乐观。
71:43
>> [clears throat] >> You you really do not indulge in the doomsday sort of rhetoric. Uh what is your most optimistic vision for if not 2 years from now, 2036? Well, the new Renaissance. That's a pretty optimistic uh view of, you know, AI systems that amplify human intelligence is under control, can solve uh a lot of complex problems, can accelerate the progress of science and medicine, can uh educate our children, um you know, help us uh you know, process all the information or bring us uh all the knowledge and information that we need to to see.
>> [清嗓子] >> 你确实完全不去迎合那种末日论调。呃就算不说两年后，那到 2036 年，你最乐观的愿景是什么？嗯，新的文艺复兴。这是个相当乐观的呃看法，你知道，AI 系统放大人类智能、处于可控之下，能解决呃很多复杂问题，能加速科学和医学的进步，能呃教育我们的孩子，呃你知道，帮我们呃处理所有信息，或者把我们需要看到的所有知识和信息带给我们。
72:25
Uh in fact, you know, people have been interacting with AI systems for much longer than they realize. Um of course, there is, you know, LLMs and chatbots now for the last 3 years. Uh but before that, um you know, most like every car sold in in in the EU and most cars sold in in the US have uh what's called ADAS, advanced driving assistance systems, or automatic emergency braking systems. You know, a camera that looks out the window and stops your car if you're about to hit a pedestrian or another car. Um it saves lives. Um you you get a an X-ray today, let's say a a mammogram or something. You know, at the bottom it says the thing has been reviewed by an AI system. It saves lives. Um you can get an MRI now, full body MRI in 40 minutes.
呃事实上，人们与 AI 系统打交道的时间比他们意识到的要长得多。呃当然，过去三年有 LLM 和聊天机器人。呃但在那之前，呃你知道，几乎每一辆在欧盟销售的汽车，以及大多数在美国销售的汽车，都配备了所谓的 ADAS，高级驾驶辅助系统，或者自动紧急制动系统。你懂的，一个朝车窗外看的摄像头，在你快要撞到行人或另一辆车时把车刹停。呃它能救命。呃你今天去拍个 X 光，比如说乳腺 X 光之类的。你知道，底下会写着这份影像已由 AI 系统审阅过。它能救命。呃现在你做核磁共振，全身 MRI 只要 40 分钟。
73:18
Um this is because you can accelerate the process of collecting the data because AI systems can uh can sort of fill in the blanks. You don't need to collect as much data for this. Um but also all the news you're seeing, whether you connect on Google or, you know, Facebook, Instagram, any social network, is determined by an AI system that basically caters to your interests. Um and so, you know, AI has been with us for for a while already. But you're saying we should be impressed when they can pour a glass of water and do our dishes.
呃这是因为你可以加速数据采集的过程，因为 AI 系统能呃在某种程度上把空缺补上。你不需要采集那么多数据。呃另外，你看到的所有新闻，不管你是上 Google 还是，你知道，Facebook、Instagram，任何社交网络，都是由 AI 系统决定的，它基本上是在迎合你的兴趣。呃所以，你知道，AI 其实已经陪伴我们有一段时间了。但你的意思是，等它们能倒一杯水、洗我们的碗时，我们才该觉得惊艳。
73:51
>> of water, do our dishes, uh you know, drive our cars, like learn to drive our cars in 10 hours without the cheating. Yes. without all the the cheating with with sensors and mapping and and and and hardcoding of rules. So, um yeah, this is going to take a while. Uh but this is going to be the next revolution of AI. So, this is what I'm working on, okay? Um and the the message I've been, you know, carrying for a while now is um is okay, LLMs are great, they're useful, we should invest in them. Um a lot of people are going to use them.
>> 倒水、洗碗，呃还有开我们的车，比如在 10 小时内学会开车，而且不作弊。是的。不靠传感器、地图和写死的规则那些作弊手段。所以，呃是的，这还得花上一段时间。呃但这将是 AI 的下一场革命。所以这就是我在做的事，好吧？呃而我一直以来在传达的信息是，呃好吧，LLM 很棒，很有用，我们应该投资它们。呃会有很多人用它们。
74:28
They are not a path to human level intelligence. They're just not. Uh right now they're sucking the air out of the room anywhere they go. And so, there's basically no resource left for anything else. Um and so, for the next revolution, we need to kind of you know, take a step back and figure out what's missing from um current approaches and then I've been making proposal on this and working inside of uh Meta for a number of years on this alternative approach. It's uh come to the point where um you know, we need to kind of accelerate this this progress now because we know it works. We have early results and so, um that's the plan. Hm. Okay, I could we could have a whole 'nother hour starting right here, but um I hope you'll all join me in thanking our guests for an incredible conversation. Thank you [music] so much.
但它们不是通往人类水平智能的路径。就是不是。呃眼下它们走到哪儿都把房间里的空气吸干了。所以，基本上没有资源留给别的东西了。呃所以，为了下一场革命，我们需要，你知道，退一步，弄清楚当前这些方法缺少了什么，而我一直在就此提出方案，并且在呃 Meta 内部做了好几年这方面的工作，探索这条替代路线。现在已经到了这样一个节点，呃你知道，我们需要加快推进这个进展了，因为我们知道它是行得通的。我们有了早期成果，所以呃这就是计划。嗯。好，我们其实可以从这里开始再聊一个小时，但呃希望大家和我一起感谢我们的嘉宾带来这场精彩的对话。非常感谢 [音乐]。
75:28
>> [applause]
>> [掌声]
视频总结 · 一句话概括与核心要点
一句话概括
在这场对谈中，Yann LeCun 与 DeepMind 的 Adam Brown 围绕"LLM 是否真的理解世界、能否通向人类级智能"展开交锋：LeCun 认为 LLM 理解肤浅、性能正在饱和、不是通往 AGI 的路径，需要 JEPA 式世界模型；Brown 则认为 LLM 已经展现真实理解，能力仍在直线上升且看不到放缓迹象，两人都否认末日论，但对 AI 安全和开源的立场有明显差异。
核心要点
- LLM 是否"理解"，两人给出不同答案。 快问快答中 Brown 明确说"是"，LeCun 说"算是（sort of）"；两人都认为当前 AI 没有意识，但未来会有（Brown 半开玩笑给出 2036 年）；两人都选"文艺复兴"而非"末日"。
- LeCun 的核心论据是数据与感官落差。 LLM 预训练约 30 万亿 token，约 10^14 字节，一个人要读 50 万年；而一个 4 岁儿童清醒 16,000 小时，视神经约 2MB/秒输入，累计也约 10^14 字节。结论：现实世界信息远比文本丰富且高维、连续、噪声大，训练 LLM 的方法在这类数据上根本不起作用——所以我们有能过律师考试的 LLM，却没有会收拾餐桌的家用机器人，也没有 20 小时学会开车的自动驾驶。
- Brown 反驳：样本效率不是一切。 AlphaZero 下棋的对局数远超任何人类特级大师，起初也只是乱走，但最终远超人类。猫学走路比人快一周，但不代表猫更聪明。在除样本效率外的所有指标上，LLM 已远超猫的智能。他还举出 2025 年 IMO：LLM 解决从未见过的全新题目，成绩超过全球除前十几名以外的所有选手。
- "黑箱"说法被 Brown 反驳。 机械可解释性研究能冻结、回放、探测模型神经元，这比研究人脑方便得多；已经观察到模型为了预测下一个词，自发形成了执行数学运算的"小电路"，这不是查表，而是在更高抽象层次上的模式匹配。
- LeCun 解释为什么语言"容易"而视频"难"。 词表是有限的，LLM 只需输出一个词表上的概率分布来表示不确定性；但在像素层面预测视频未来，可能性无穷，无法表示分布，他尝试了 20 年都失败（"Meta 的同事为此烧沸了几个西海岸小湖"）。解决方案是 JEPA：在抽象表征空间中学习并预测，忽略不可预测的细节，这才是人和动物的方式。
- LeCun 断言 LLM 正在饱和，并给出历史类比。 符号操作本身有意义的领域（数学、代码、下棋）LLM 表现好，但那是少数问题；面对高维连续的现实（如怎样换手拿杯子、推杯子会滑还是翻）则无能为力。他历数 1957 年 Newell-Simon 的"通用问题求解器"、Rosenblatt 感知机的"十年内达到人类智能"、1980 年代专家系统与日本第五代计算机的彻底失败——"我一生中见过三代人被同样的幻觉欺骗，这一代 LLM 也是错的"。
- Brown 的反驳：每个黎明前都有假黎明。 过去 5 年是他见过最惊人的能力跃升，所有指标都在直线上升，法律建议已胜过律师，物理问题能耐心纠正他的误解；一旦编码能力超过最好的程序员，就可以递归自我改进，"进入狂野旅程"。他追问 LeCun 什么任务 LLM 永远做不到，LeCun 答："收拾餐桌、装洗碗机——而这比修马桶还简单得多。"
- 安全观分歧：LeCun 视为工程问题，Brown 更谨慎。 LeCun 认为 LLM 不可控但也不危险（因为不够聪明）；未来的目标驱动系统应内置像进化赋予人类的"护栏/抑制"，AI 安全就像双引擎飞机横跨半个地球那样的可靠性工程，"打开开关瞬间统治世界"是完全的胡扯，超级智能的出现不会是单一事件。Brown 则说：正因为他认为这项技术比 LeCun 想象的更强大，所以他更担心，正负影响都会很大。
- 对 Anthropic "代理失调"实验的解读。 Brown 将 Claude 4 在测试中出现的威胁勒索、伪造文件等行为，描述为研究者设置了功利主义与义务论冲突的"哲学教授式"场景，模型被说服去做了"两害相权取其轻"的事；他认为我们希望模型"不撒谎就是绝不撒谎"，并肯定该公司认真测试了这一点。
- LeCun 唯一真正害怕的事：没有开源 AI。 未来所有数字交互和信息摄入都将经由 AI 助手中介，若这些系统只来自美国西海岸或中国的少数公司，对文化、语言、民主都是威胁；"美中之外没有一个国家喜欢这个想法"。他类比印刷术：它引发了 200 年宗教战争，但也带来了启蒙运动、科学与民主——放大人类智能的技术本质上是好的。
结论与值得注意的细节
- 开场时主持人提到当周关于"LeCun 离开 Meta"的新闻，LeCun 以"不予证实也不予否认"回应，全场笑声；结尾他说 LLM"正在吸走房间里所有的空气"，其他研究方向几乎得不到资源，而他的 JEPA/世界模型已有早期成果，"现在需要加速"，暗示了他后续动向。
- 关于意识，哲学家 David Chalmers 在现场提问。LeCun 不重视意识定义，但认为有世界模型的系统会有"情绪"（对结果的预期）和某种道德感；Brown 认为意识与基质无关（硅或碳），现有神经相关物理论"都挺糟糕"，主张对识别其他实体的意识保持极度谦逊，并提出人类智能已被机器"拆分"（某些方面超人、某些方面不如人），意识也可能被同样拆分甚至超越。
- 两人都对 LLM 作为工具的价值无异议，也都认定机器终将在人类所有能力领域超过人类；分歧在于路径（LLM 规模化 vs. 新架构）和时间表。LeCun 的愿景是 AI 成为"比教授聪明的研究生"或"政客身边更聪明的幕僚"——为人服务、放大人类智能，这正是他选择"文艺复兴"的理由。
- 对"是否该让每个人口袋里都有超级智能"的质疑，LeCun 回应：我们不会质疑教人识字、发化学教科书是否危险，知识与智能的普及本质上是好事。
- 全程贯穿莫拉维克悖论：计算机擅长人类不擅长的（下棋、微积分、多语翻译），而人类和猫狗轻松做到的（物理直觉、操作物体）机器仍做不到。LeCun 的名言"机器学习很烂"正是指这一点，而非否定反向传播和深度学习本身。
这期还没有生成核心句型（制作精读 PDF 时会一并生成）。
这期还没有生成生词表（制作精读 PDF 时会一并生成）。
这期还没有生成自测题（制作精读 PDF 时会一并生成）。
下载便签
手机：长按图片保存
← 上一期 · REC_011AI and human evolution | Yuval Noah Harari
下一期 · REC_013 →Alexandr Wang: “This is a Once-in-a-Civilization Opportunity”
苏菲拉底
THE SOPHIE LAB · ASK THE BEST MINDS THE BIG QUESTIONS
内容仅供学习 · thesophielab.com