视频库 / REC_025
--:--
点击播放，这里会跟随视频显示当前句的中英字幕。
第 25 期 · 回应 新·03「当通用模型持续变强，创业者还能在哪里建立持久优势？」
Jeff Dean: The 1% Rule for Building in AI
节目发布 2026-07-30 · Y Combinator章节 · 点击跳转视频
0:07
回看预测：AI 已是初级工程师
2:40
历史重演：推理硬件的内存时刻
6:00
餐巾纸算出来的 TPU
10:25
AI 时代的延迟与能耗数字
16:07
上下文工程与 skills 实践
22:01
长时 agent 为何脱轨与对策
25:37
1% 法则：创业公司在哪能赢
31:07
规格说明与选题品味
36:10
思想实验：不可靠晶体管与 MapReduce
42:03
自动化科学方法与快速评估器
47:15
蒸馏被拒与职业路径选择
53:10
组队原则与留给后来者的问题
本期讲者
杰夫·迪恩Google 首席科学家，1999 年加入 Google，主导或参与了 MapReduce、Bigtable、TensorFlow、TPU 和 Gemini 等系统，与 Hinton、Vinyals 合著知识蒸馏论文。
主持人AI Ascent 类创业者峰会的主持人，面向数千名未来创业者，围绕系统工程与创业选题向 Dean 提问。
01回看预测：AI 已是初级工程师
0:07
All right. Should we go Should we get started, Jeff? >> Sure. Sounds great. >> All right. Jeff, welcome. And again, thank you so much for being here. Especially I just got a cold and thank you for being here. >> Yeah, I'm afraid I've lost my voice. I don't normally sound quite like this, but we'll we'll do what we can. >> So, um, you built map reduce, big table, tensorflow, the TPU, Gemini. We could spend a whole hour on all the things you've done, but what I love is that you're still making bold predictions in public. Last year, yes, last year in May 2025 at AI Ascent, you said that AI is at the level of a junior engineer.
好的。我们要不要开始了，Jeff？是的，听起来不错。好的。Jeff，欢迎你。再次感谢你能来。尤其是我刚感冒了，谢谢你能来。是啊，我这嗓子恐怕是哑了。我平时说话不是这个声音，不过我们尽力吧。那么，你做过 MapReduce、Bigtable、TensorFlow、TPU、Gemini。光是你做过的这些事，我们就能聊上一整个小时，但我特别喜欢的一点是，你到现在还敢在公开场合做大胆的预测。去年，对，就是去年，2025 年 5 月在 AI Ascent 大会上，你说 AI 已经达到了初级工程师的水平。
0:50
That was about a year ago. It's been How close are we to that prediction? Yeah, I mean I feel like uh the models have been getting a lot better at sort of agent-based longer running coding tasks and it seems pretty clear that they are now actually pretty capable and depending on exactly your definition of of junior engineer it seems pretty spot-on I would say. >> What did you underestimate from that prediction? Um I mean I think the the ability to do more and more complex tasks has been growing faster than I thought. Um and I also think uh outside of coding these these agent-based systems are are really starting to shine in other domains and I I think uh you know uh that's that's going to be an important trend in the future.
那大概是一年前了。到现在，我们离那个预测有多近了？是啊，我觉得这些模型在那种基于 agent 的、长时间运行的编程任务上进步了很多，而且现在看来相当明确，它们确实已经挺有能力了。当然这取决于你对“初级工程师”的具体定义，但我觉得那个说法基本上是相当准确的。那个预测里，你低估了什么？嗯，我觉得处理越来越复杂任务的能力，增长得比我想象的要快。另外我还觉得，在编程之外，这些基于 agent 的系统在其他领域也真的开始大放异彩了，我觉得……知道吧，呃，这在未来会是一个很重要的趋势。
1:44
>> So [snorts] give us another bold prediction. What do you think is going to be the 2027 edition? >> Uh I think you will see a lot more automation of uh ML systems themselves. um basically getting ML systems to improve their capabilities by running lots of experiments, breaking things down into subpros, you know, running those subpros in a tight automatic experimentation loop, putting the results together and being able to then uh you know, get some improved system uh out from that uh sort of fully automated problem decomposition and and automated experimentation that I think that's going to be really exciting. M >> I think that also applies not just to ML but also to other fields of science and engineering. Um basically anything where you can have a measurable objective uh I I think you can uh actually make a lot of progress these days.
那么 [吸鼻子] 再给我们一个大胆的预测吧。你觉得 2027 版本会是什么样？呃，我觉得你会看到 ML 系统本身会有更多的自动化。嗯，基本上就是让 ML 系统通过运行大量实验来提升自己的能力，把问题拆解成子问题，你懂的，把这些子问题放进一个紧凑的自动实验循环里跑，再把结果汇总起来，然后就能，呃，你懂的，得到一个改进后的系统，呃，从那种完全自动化的问题拆解和自动化实验中得出结果，我觉得这会非常令人兴奋。嗯，我觉得这不仅适用于 ML，也适用于科学和工程的其他领域。嗯，基本上任何有可衡量目标的领域，呃，我我觉得如今你都可以，呃，取得很大的进展。
02历史重演：推理硬件的内存时刻
2:40
>> Now let's go back to a little bit in history. Back in way back in 2001 Google search used to run on hard drives. >> Yep. And you and Sanjay did the math and realized that at some point the whole search index would finally fit in all of the RAM of all the computers you had running and you made that radical realization and you basically in few days with Sanjay shipped in production a whole new search version that worked in RAM rather than hard drive and that was the thing that got Google to be so fast. Google searches.
那我们回顾一下历史。早在 2001 年，Google 搜索还是跑在硬盘上的。是的。你和 Sanjay 算了一笔账，意识到到某个时候，整个搜索索引最终可以完全装进你们所有在运行的机器的内存里，你们做出了那个激进的判断，然后基本上在几天之内就和Sanjay 一起把一个全新的搜索版本推上了生产环境，它跑在内存里而不是硬盘上，正是这件事让 Google 搜索变得那么快。
3:21
>> So history tends to remix. What is the it fits the memory moment right now in 2026 that everyone in this room is still should be thinking about and designing? >> Yeah. Yeah, I mean it's a little different, but I think uh you're going to see more and more uh uh high performance and um low energy uh inference hardware systems because I think everyone is now realizing that inference is the key to making you know these agent-based systems be available to more and more people and that latency is really important and that specialization of the hardware is a really key way you can make uh things that are more energy efficient and lower latency than more general purpose uh computational devices like say GPUs or TPUs >> because I think all everyone here is used to waiting for responses on on models. [clears throat] So >> waiting is no fun >> master speed.
所以历史往往会重新混搭上演。现在是 2026 年，什么才是当下真正契合这个时刻的记忆（memory）问题，是在座各位仍然应该去思考、去设计的？>> 是的。是的，我是说这个话题稍微有点不一样，但我觉得你会看到越来越多高性能、低能耗的推理硬件系统，因为我觉得现在大家都意识到，推理才是让这些基于智能体的系统能够被越来越多的人用上的关键，而且延迟真的非常重要，硬件的专用化是一个非常关键的手段，能让你做出比通用计算设备更节能、延迟更低的东西，比如说 GPU 或者 TPU 这类更通用的计算设备。>> 因为我觉得在座的各位都已经习惯了等待模型的响应。[清嗓子] 所以 >> 等待可不好玩 >> 速度为王。
4:27
So you're saying what if we don't have to wait anymore? >> Yeah. I mean, I think we'll imagine what you could do with something where the latency is, you know, 50x better. >> Interesting thought. Now, what's one assumption that perhaps 6,000 people in this room hold that's already false about AI? >> Yeah. Uh, that's that's a good question. I mean I think um probably one thing is people don't quite realize how possible it is to have you know agent-based systems that can run not just for an hour or two hours on a problem you care about but for some problem domains and with highly capable models underlying them you can get them to run for days or weeks and do really really complicated tasks and I think that's you know starting some people are starting to see inklings of this but I don't think everyone has really internalized this and that's going to be really uh a pretty big deal.
所以你的意思是，如果我们不再需要等待了会怎样？>> 是的。我是说，想象一下如果延迟能好上 50 倍，你能用它做些什么。>> 有意思的想法。那么，这个房间里大概 6000 人所持有的、但其实关于 AI 已经不成立的一个假设是什么？>> 嗯，这是个好问题。我觉得，可能有一点是：大家还没完全意识到，基于智能体的系统其实已经可以运行得不只是在你关心的问题上跑一两个小时，而是在某些问题领域里，配上能力足够强的底层模型，你可以让它们连续跑上几天甚至几周，去完成非常非常复杂的任务。我觉得现在已经有一些人开始看到这方面的苗头了，但我不认为所有人都真正消化了这一点，而这将会是一件相当大的事。
5:28
>> What's a particular task that you have run that has run for weeks? What what was it? What did the tell what did you tell the agents to solve? Yeah, I mean I think uh you can tell agents to uh go off and implement um you know completely new versions of software in different programming languages that might be you know have better safety properties or better performance properties uh that and then then they can go off and and actually do that in a you know pretty serious way. >> That's pretty cool.
>> 有没有哪个具体的任务是你跑过、而且跑了好几周的？是什么任务？你当时让智能体去解决什么问题？是的，我觉得你可以让智能体去用不同的编程语言，把某个软件完全重新实现一个新版本，可能具备更好的安全特性或者更好的性能特性，然后它们就真的可以相当认真地把这件事做出来。>> 那挺酷的。
03餐巾纸算出来的 TPU
6:00
[clears throat] Now, one thing that you've been very well known for is you're really good at napkin math. Sounds funny. So, one of the stories about you is that back in uh 2013 when speech recognition started to work at Google, you did the nap napkin math where if every Google user used their phone and talked to it and used the speech recognition system for three minute just three minutes a day, you found that the system requires a Google server. you would have to double the fleet which would be really really expensive just to do speech translation.
[清嗓子] 你有一点特别出名，就是你非常擅长「餐巾纸计算」（napkin math）。听起来挺有意思。关于你的一个故事是，早在 2013 年，当语音识别在Google 开始真正跑通的时候，你做了个餐巾纸计算：如果每个 Google 用户都用手机跟它说话、每天使用语音识别系统哪怕只有三分钟，你发现这个系统需要的 Google 服务器数量，会让你不得不把整个服务器集群翻一倍，光是为了做语音识别就贵得离谱。
6:39
>> Yeah. >> And instead you basically built a custom ship and that was the origin story of the TPU. >> Yeah. Yeah. I mean I I sort of had done you know we were starting to see really good uh quality results on this the sort of deep learning based speech systems uh speech models we were training. um but they were computationally expensive compared to the old speech system but they haved the error rate. So that was like the equivalent of 20 years of advances in speech recognition in just a few months of like fiddling with the model and getting scaling it up a bit and getting better data. And so we started to get worried that if speech worked a lot better, people would use it more. And so that that back of the envelope calculation was really about that like well what if people start start to use speech recognition more to dictate emails or to talk to their phone or whatever. Um and yeah it turned out that um we realized that we needed some better solution than running on CPUs at the time. And so we came up with TPUs
>> 是的。>> 于是你基本上做了一颗定制芯片，这就是 TPU 的起源故事。>> 是的，是的。我是说，当时我们开始在这些基于深度学习的语音系统、我们训练的语音模型上，看到质量非常好的结果。但跟老的语音系统相比，它们的计算开销很大，不过错误率降了一半。所以那基本上相当于语音识别领域 20 年的进展，却只用了几个月——就是摆弄一下模型、把规模稍微扩大一点、再拿到更好的数据。于是我们开始担心，如果语音效果好这么多，大家就会用得更多。所以那个粗略估算其实就是在算：如果大家开始更多地用语音识别来口述邮件、跟手机说话之类的，会怎么样。结果我们意识到，当时在 CPU 上跑确实需要一个更好的方案。于是我们搞出了 TPU，它本质上是高度专门化地
7:47
which are sort of very specialized for essentially low precision dense linear algebra which is at the heart of nearly all of the modern machine learning algorithms we we use today. And um if you build a specialized chip for low precision dense linear algebra and can't do anything else that turns out to be really useful for machine learning inference uh even though it can't run Chrome or Word or whatever. Uh, and so that system produced a chip a couple years later that was uh 30 to 80 times more energy efficient than CPUs and GPUs of the day and also much much lower latency like 20 to 30x lower latency >> which is incredible what the foundation that TPU has become today. No way you would have predicted that TPU would be so foundational now with transformer architecture which was invented way later before you actually invented the TPU. Yeah, I mean that's sort of why we built a general purpose linear algebra system, which is what a TPU is really.
做低精度稠密线性代数，而这正是我们今天使用的几乎所有现代机器学习算法的核心。如果你造一颗只能做低精度稠密线性代数、别的什么都干不了的专用芯片，结果它对机器学习推理特别有用——尽管它跑不了 Chrome 或者 Word 之类的东西。所以那个项目在几年后产出了一颗芯片，能效比当时的 CPU 和 GPU 高出 30 到 80 倍，延迟也低得多，大概低 20 到 30 倍。>> 这太不可思议了，看看 TPU 今天已经成为多么重要的基础设施。你当时绝不可能预料到，TPU 在 Transformer架构出现之后会如此关键——而这个架构是在你发明 TPU 很久之后才被提出来的。是的，这某种程度上就是我们为什么造了一个通用的线性代数系统，TPU 其实就是这么个东西。
8:48
Um, because we knew ML algorithms were still evolving and you didn't want to over specialize, but you wanted to specialize enough that you got the dramatic performance benefits of we could have very big multiplier units. uh we could have you know high-speed memory we could have high-speed interconnect or later TPUs that like brought many many chips to bear on the same problem efficiently and u you know we've continued to scale those up and and improve their performance uh for over many many generations now >> incredible napkin math so >> what's good >> napkins are good >> so actually what's a good napkin math that everyone here who wants to be a future founder should run tonight to potentially build something as consequential as the TPU.
因为我们知道机器学习算法还在演进，你不想过度专用化，但又想专用到足以获得显著的性能收益——我们可以放很大的乘法单元，可以有高速内存，可以有高速互连；后来的 TPU 还能把非常多的芯片高效地投入到同一个问题上。我们也一直在把它们的规模往上扩、把性能往上提，到现在已经很多很多代了。>> 了不起的餐巾纸计算。>> 挺好的 >> 餐巾纸是个好东西 >> 那么实际上，对在座每一位想成为未来创业者的人来说，今晚可以做一个什么样的餐巾纸计算，去有可能造出像 TPU 那样有分量的东西？
9:39
>> Yeah, I mean, uh, it's always hard to say. Um, [clears throat] I think, uh, think about what problems you see in whatever it is you're thinking about, what what bottlenecks you see, and are there very different ways of thinking of the solutions to some of those problems that would get you, you know, an order of magnitude or two orders of magnitude better uh, performance or capability or whatever it is. Um, you know, because sometimes if you just squint at a problem and you think about not necessarily being anchored on exactly how that problem is solved today, but how you would solve it from first principles, you can come up with really good ideas that are, you know, maybe not what other people are thinking about.
>> 嗯，这个总是很难说。我觉得，想一想在你正在琢磨的那个领域里，你看到了哪些问题、哪些瓶颈，以及对其中某些问题，有没有非常不一样的解决思路，能让你在性能、能力或者别的什么指标上，拿到一个数量级甚至两个数量级的提升。因为有时候，如果你眯着眼睛看一个问题，不一定被今天这个问题是怎么解决的所束缚，而是思考如果从第一性原理出发你会怎么解，你就能想出一些非常好的点子，而这些可能正是别人没在想的。
04AI 时代的延迟与能耗数字
10:25
>> That's a good tip. [clears throat] >> No. Um, for everyone here who doesn't know, years ago, Jeff wrote a very famous list called the latency numbers. every engineer should know and these are numbers around for example how long a cache miss takes uh disk seek a network package traveling let's say from California to Netherlands um lots of numbers like this about distributed systems and systems engineering [clears throat] >> and it's been sort of taped and become the bible for a lot of distributed systems engineers >> okay yeah >> now fast forward that list is up for an update give us the AI edition for now 2026.
>> 这是个好建议。[清嗓子] >> 对了，可能有人不知道，很多年前 Jeff 写过一份非常有名的清单，叫做「每个工程师都该知道的延迟数字」。这些数字包括比如说一次缓存未命中要多久、磁盘寻道要多久、一个网络数据包从加州传到荷兰要多久，诸如此类关于分布式系统和系统工程的很多数字。[清嗓子]>> 这份清单基本上被贴在墙上，成了很多分布式系统工程师的圣经。>> 好的，是的 >> 现在快进到今天，这份清单该更新了，给我们讲讲 2026 年的 AI 版本吧。
11:09
>> Yeah, I mean I think if you looked at what is important in AI systems these days, you would want to know things like the bandwidth between you know your main memory system on your accelerator to the onchip memory to the um you know the multiplier unit or whatever. You want to know how much energy does it take to do a single multiplier operation. um you know uh what is the interconnect bandwidth between chips and how much does that uh how how many chips can you connect with that bandwidth and then if you go beyond that domain like what is the fall off in in uh network bandwidth when you need to talk to 10,000 strips instead of instead of uh 500 or something I think these are all really important numbers to to learn and and really affect how you think about solving particular kinds the problems.
>> 是的，我觉得如果你看看当下 AI 系统里什么是重要的，你会想知道这些数字：比如加速器上主存到片上内存、再到乘法单元之间的带宽是多少。你会想知道做一次乘法操作要消耗多少能量。还有芯片之间的互连带宽是多少，用这个带宽你能连接多少颗芯片；再往外走一层，当你需要跟 10,000 颗芯片通信而不是 500 颗时，网络带宽会掉到什么程度。我觉得这些都是非常重要、值得去掌握的数字，而且它们会实实在在地影响你思考如何解决某类问题。
12:04
>> H [clears throat] and one interesting thing that I've heard you talk about is that nowadays the unit that you measure everything is energy. >> Yeah. >> You pointed out that doing a calculation or math costs about one pico. Uh but moving the data and doing data IO costs thousand times that. >> Yeah. Just bringing it in from HPM on an accelerator into the processor so it can actually compute on it. Yep. That gap kind of quietly decides what products are possible and how these algorithms in AI are built. So what are the kinds of problems that founders keep calling model problems but are in fact actually energy or data IO problems?
>> 嗯 [清嗓子] 还有一件有意思的事，我听你讲过：现在你衡量一切的单位是能量。>> 是的。>> 你指出过，做一次计算大约消耗一皮焦（pico）。但搬运数据、做数据 IO 的成本是它的一千倍。>> 是的。就是把数据从加速器上的 HBM 搬进处理器，好让它真的能拿来计算。没错。这个差距悄无声息地决定了哪些产品是可行的、AI 里这些算法是怎么设计的。那么有哪些问题，是创业者一直称之为「模型问题」，但实际上是能耗或者数据 IO 问题？
12:51
Yeah, I mean I think the the example you raised of a thousandx difference in bringing mo moving data versus actually computing on it uh in in terms of energy is is a pretty significant one and it shapes a lot of aspects of what we do in machine learning. Um because if you didn't have that thousandx difference then you know you wouldn't have to do batching but you have to do batching of you know many examples or maybe many tokens at once in order to amortize that data movement [clears throat] so that you can uh you know not pay a thousandx slowdown but pay a 1000x divided by batch size uh energy cost. Um and you know for for really low latency batching is not really very good. Um so I think these [clears throat] kinds of things and the energy uh behind various decisions in the computer hardware we use really affects a lot of decisions we make in building higher level systems.
是的，我觉得你提到的那个例子——搬运数据和真正在数据上做计算之间在能耗上有一千倍的差距——是相当重要的一点，它塑造了我们在机器学习里做的很多事情。因为如果没有这一千倍的差距，你其实就不需要做批处理（batching）了；但现在你必须把很多样本、或者很多 token 一次性打成一批，才能摊薄那个数据搬运的开销 [清嗓子]，这样你付出的就不是一千倍的代价，而是一千倍除以批大小的能耗代价。而对于要求极低延迟的场景，批处理其实并不太好。所以我觉得这类事情，以及我们所用的计算硬件里各种取舍背后的能耗因素，确实会深刻影响我们在构建上层系统时做出的很多决策。
13:51
>> A very concrete example is just how training models is done. There's this whole whole concept of batching the the data sets and running epochs. That's basically people perhaps may confuse that as a model problem, but it's really a systems data IO problem, right? >> Yeah. Yeah. I mean, you have to assemble batches to get better efficiency in your hardware. You know, ideally you might do batch size one training, but uh you know, it's um not as not as good in terms of efficiency. So people use re pretty large batches these days.
>> 一个非常具体的例子就是模型训练的做法。有一整套关于把数据集打成批次、跑若干个 epoch 的概念。人们可能会误以为那是个模型问题，但它其实是一个系统层面的数据 IO 问题，对吧？>> 是的，是的。我是说，你必须把数据攒成批，才能让硬件效率更高。理想情况下你可能想做批大小为 1 的训练，但那在效率上就没那么好了。所以现在大家用的批次都挺大的。
14:26
>> Do you think uh it's possible for uh I know you're you're well known for uh taking off uh on a long week or weekend and coming up with this brilliant solution. Is there such things of Jeff going and working on it for a couple weeks and [clears throat] getting batch size equals one training done. >> Yeah, I've been thinking more about inference actually. So I think inference is a pretty interesting problem because you do want very low latency. You know training you don't necessarily need incredibly low latency. Um and I think there's a lot of room for specializing hardware more for inference than we are today.
>> 你觉得有没有可能……我知道你以出名的一点是，你会抽出一个长周末或者一周跑去琢磨，然后想出某个绝妙的解法。会不会出现 Jeff 花几个星期，[清嗓子]把批大小等于 1 的训练给搞定了？>> 嗯，其实我最近想得更多的是推理。我觉得推理是个相当有意思的问题，因为你确实需要非常低的延迟。而训练不一定需要极低的延迟。我觉得在为推理做更多硬件专用化这件事上，还有很大的空间，比我们今天做的要多。
15:02
>> What are some of those interesting things that are on inference that you're really thinking a lot about? >> Um I mean just trying to minimize data movement. uh trying to think about incredibly uh low precision operations uh and maybe not supporting lots and lots of different kinds of precisions. Uh if you feel like you have a a good answer for what kinds of precision you need, maybe just build that into the hardware and and um not much else. which I think it brings down to a core [clears throat] analogy I heard from famous computer scientists that really the whole process of u AI is a big compression problem because in order to have the data to be f fully lossy and compress it and then restore it you basically need to understand it. Yeah, I mean if you truly understand the data, you should be able to compress it really well >> and now transformer architecture is basically one of the ways that has turned out to work really well.
>> 在推理方面，有哪些有意思的东西是你现在思考特别多的？>> 嗯，就是尽量减少数据搬运。还有考虑极低精度的运算，以及也许不去支持特别多种不同的精度格式。如果你觉得自己对「需要哪些精度」已经有了好答案，那也许就把它直接做进硬件里，别的就不要了。这让我想到一个核心的类比，我从一些著名计算机科学家那里听来的：其实整个 AI 的过程就是一个巨大的压缩问题，因为要把数据完全（有损地）压缩、再还原出来，你基本上必须理解它。是的，我是说如果你真正理解了数据，你就应该能把它压缩得非常好。>> 而现在 Transformer 架构基本上是被证明效果非常好的方式之一。
05上下文工程与 skills 实践
16:07
>> Yeah. Yeah, I would say >> working pretty well so far. >> Good work by my colleagues. [laughter] >> Yes. Now let's zoom out a bit. Um AI progress used to mean just better models. You could had more data trainer models with bigger parameters. But increasingly in the last years or so, it's everything around the model. Not just the model size and number of parameters or more data. It's everything around things like retrieval tools, memory, agent tools, and it might kind of get consolidated into what people call uh context engineering, right?
>> 是的，是的，我会说 >> 到目前为止表现相当不错。>> 这得归功于我同事们的出色工作。[笑声] >> 是的。现在我们把视角拉远一点。以前 AI 的进步基本就意味着更好的模型：更多数据、更大参数量的模型。但在过去这一两年里，越来越多的进步来自模型周边的一切，而不只是模型大小、参数量或者更多数据。是围绕模型的那一整圈东西，比如检索工具、记忆、智能体工具，这些可能会被整合成人们所说的「上下文工程」（context engineering），对吧？
16:42
[clears throat] >> Yeah. I mean I think uh the model is really only one piece of what you're trying to do which is build an overall system that can solve really interesting problems and that involves you know a model that knows how to use various tools. It maybe knows how to retrieve relevant information, maybe has a, you know, a history of other uh information that it has retrieved for past problems and it can put information into the context of the of the model. And the nice thing about that is that information is really clear to the model, unlike the training data the model was trained on where it's all kind of like trillions of tokens stirred together into a soup of of hundreds of billions or trillions of parameters, but it's all less clear than the actual context uh that the model sees directly for this particular problem or uses use case. And then I think being able to understand what tools are available, which ones are going to help me solve the help the model solve this next you
[清嗓子] >> 是的。我觉得模型其实只是你想做的事情中的一块，你真正要做的是构建一个整体的系统，能解决非常有意思的问题；这里面涉及一个知道如何使用各种工具的模型。它可能知道怎么去检索相关信息，可能保存着以往为其他问题检索过的信息的历史记录，然后它可以把信息放进模型的上下文里。而这样做的好处是，这些信息对模型来说非常清晰明确，不像模型训练时用的训练数据那样，是几万亿个 token 被搅在一起，糊成了几千亿甚至上万亿参数的一锅汤，但是这一切都不如模型针对这个特定问题或用例直接看到的实际上下文那么清晰。然后我认为，能够理解有哪些工具可用，哪些工具能帮我解决——帮模型解决问题的下一个阶段，如何把一个问题拆解成一连串的
17:47
know phase of the problem, how to decompose a problem into a sequence of of tool calls. Maybe trying multiple approaches to solve the problem and seeing which ones work and being able to evaluate that. you know this is the whole um you know orchestration of complex agent and multi-agent systems that I think is going to be more and more important and uh super exciting times I would say >> and I think the fun thing about this particular problem domain set is actually something that everyone in this room can actually do because before to train a model you needed incredible amount of resources incredible amount of access of to GPUs and data but for context engineering everyone here could do you have you just need the API to something like Gemini and then work on your own setup for your own retrieval your own tool calls and etc etc. So how does what are some tips for everyone here? How does everyone get better at and become exceptional at context engineering? Yeah, I mean I think uh [clears throat]
工具调用。也许尝试多种方法来解决问题，看看哪些行得通，并且能够对此做出评估。你知道，这就是复杂智能体和多智能体系统的整体编排，我认为它会越来越重要，可以说是非常令人兴奋的时代。>> 我觉得这个特定问题领域有意思的一点是，它其实是在座每个人都能做的事，因为过去要训练一个模型，你需要难以想象的资源、难以想象的 GPU 和数据的获取渠道，但对于上下文工程，在座的每个人都能做——你只需要拿到像 Gemini 这样的 API，然后去打磨你自己的一套东西，你自己的检索、你自己的工具调用等等。那么，对在座各位有什么建议？大家怎样才能变得更擅长，甚至在上下文工程上做到出类拔萃？是啊，我想（清嗓）一个很好的办法就是去用
18:50
a really good way to do it is to use these models and and sort of harnesses and tools and so on to try to solve problems and then some sometimes you can actually see where the models are failing. And often you can actually make the model work better and succeed at that kind of problem by not just adjusting the model parameters which is hard to do from the outside but from you know creating better guidelines for the model you know writing skills for the model to know how to use different tools that would be incredibly useful for solving this particular class of problem. And I think as you do that, you end up on this kind of improving self-improving of the setup that you're trying to use to to solve things. Uh, and you know that that's a really good way to get better at understanding what what additional information the model would want in order to become more capable.
这些模型，以及各种脚手架（harness）、工具等等，去尝试解决问题，然后有时候你能真切地看到模型是在哪里失败的。而且往往你其实可以让模型表现更好、在那类问题上取得成功，做法不是去调整模型参数——那从外部很难做到——而是为模型编写更好的指引，为模型编写 skills，让它知道如何使用各种工具，这些对解决这一类特定问题会非常有用。我认为随着你这样做，你最终会进入一种不断改进、自我改进的循环，改进你用来解决问题的这套配置。呃，而且你知道，这真的是一个很好的途径，让你更好地理解模型还想要什么额外信息才能变得更有能力。
19:46
>> Can you give an example of uh some context engineering you personally have done? um I don't know skills you wrote tools that really made a huge different in your in your workflow. Yeah, I mean I guess uh Sanjay and I were working a few weeks ago and we you know we often do some amount of like uh performance improvement for very low-level libraries and we have a microbenchmark library we've written at Google where you can write microbenchmarks of how how long different kinds of operations take or how long does it take to populate this data structure whatever and sometimes those data structures are used on millions of processes across Google. So, it's actually pretty important to make sure they're high performance. And so, you can write microbenchmarks. Um, but then without an agent-based system, what you usually do is you measure what the current performance is on some benchmarks you care about. You make some modifications to improve the performance you hope. Then you rerun the the benchmarks, see where things improved.
>> 你能举个例子吗，说说你个人做过的上下文工程？比如你写过的 skills、真正对你的工作流产生巨大影响的工具。是啊，我想想，几周前 Sanjay 和我在做一些工作，你知道我们经常会为一些非常底层的库做性能优化，我们在 Google 写了一个微基准测试库，你可以用它来编写微基准，测量各种操作耗时多久，或者填充某个数据结构要多久，诸如此类，而有时候这些数据结构会被 Google 内部数百万个进程使用。所以确保它们的高性能其实相当重要。因此你可以写微基准。嗯，但如果没有基于智能体的系统，你通常的做法是：先测量你关心的那些基准当前的性能，然后做一些你希望能提升性能的修改，接着重新跑基准，看看哪里有改善。
20:48
Um, you run a maybe a broader set of benchmarks, measure the cache footprint of things. And so we wrote a skill that basically taught the model how to do most of those things in in var in various sequences so that it could actually you know do self-improving uh benchmark measurement benchmark improve you know code changes measure the performance improvement and then iterate on that and that that seemed to work uh pretty well for some kinds of problems. And it really just is us giving the approach we would use as people to the model in a form that it could use.
嗯，你可能会跑一套更广泛的基准，测量缓存占用之类的。所以我们写了一个 skill，基本上就是教模型如何按各种顺序完成这些事情，这样它就能真正做到自我改进——跑基准测量、改进基准、做代码修改、测量性能提升，然后不断迭代，而这在某些类型的问题上效果似乎相当不错。这本质上就是我们把人会采用的方法，以模型能用的形式交给它。
21:23
>> Wow, that seems very impressive. So you're saying you have this skill that if someone got access to it, it could do perform optimizations like Jeff Dean. Seems like the world would love this and is worth infinite amount of money to someone have access to this. >> Oh. Uh we actually published a document maybe a few months ago called performance hints that Sanjay and I wrote that's like a 30-page document about you know various kinds of performance tricks and some people have taken that and then given it in summarized form to various models and seen that they that model can now get you know better at uh per reasoning about performance issues in code.
>> 哇，这听起来太厉害了。所以你是说，你有这么一个 skill，如果有人能拿到它，就能像 Jeff Dean 一样做性能优化。感觉全世界都会想要这个，能拿到它对某些人来说值无限多的钱。>> 哦。其实我们几个月前发布过一份文档，叫《performance hints》，是 Sanjay 和我写的，大概 30 页，讲的是各种各样的性能技巧，有些人把它拿去做成摘要形式喂给各种模型，然后发现模型在推理代码中的性能问题上确实变得更强了。
06长时 agent 为何脱轨与对策
22:01
>> So you heard it all here. You could actually get your own optimize your own code like Jeff Dean if you take this this paper that you published when performance hints. Yep. >> It's all free available, so you should all try it. >> Very cool. >> Yeah. >> Now, you're talking about agents. Um, everyone here is probably building one or built one at some point. And I'm sure everyone has seen your agent go off the rail at perhaps like step 30 or 40. Like agents are great for like up to step, I don't know, 10 or something and then gets shaky at step 50. What do you think is the constraint today? Is it like context evaluators or just errors that compound because it's basically a openloop system?
>> 所以大家都听到了。如果你拿上这份你们发布的《performance hints》，你其实真的可以像 Jeff Dean 一样优化自己的代码。没错。>> 而且都是免费公开的，所以大家都该去试试。>> 非常酷。>> 是的。>> 那我们来聊聊智能体。在座各位大概都在做智能体，或者曾经做过。我相信每个人都见过自己的智能体在大概第 30 步或第 40 步的时候跑偏。就是说智能体在大概前十步左右表现很好，到第 50 步就开始不稳了。你认为今天的瓶颈是什么？是上下文、评估器，还是因为它本质上是个开环系统而导致误差不断累积？
22:41
>> Yeah, I mean obviously we want agents to be able to run for very long periods of time because that's how they're going to solve more and more complicated problems. Um but as you as you observe today, you know, they sometimes stop working after, you know, 10 10 interactions with the tools and so on. Um, and sometimes that's because the model is trying to do something it doesn't have a lot of experience doing. So it's been trained on a whole set of things and as soon as you get a little bit off the distribution of things it knows how to do then like most machine learning models it will you know its performance will suddenly will start to degrade and the farther you get off the comfort zone of what it knows how to do the the more likely it is to to not work as well. Um so there's a bunch of things you can do. So one is you know give the model skills and hints that kind of tend to keep it in in the uh sort of more brightly lit path of things it does know how to do. Um, I think you know having
>> 是啊，显然我们希望智能体能长时间持续运行，因为只有这样它们才能解决越来越复杂的问题。但正如你今天观察到的，它们有时候在和工具交互十来次之后就不工作了。嗯，有时候是因为模型在尝试做一件它没什么经验的事。它在一整套事情上被训练过，而一旦你稍微偏离它知道怎么做的那个分布，那么就像大多数机器学习模型一样，它的表现会突然开始下降，而且你越是偏离它熟悉的舒适区，它就越可能表现不佳。嗯，所以你可以做很多事情。一是给模型 skills 和提示，让它倾向于待在它确实知道怎么做的那条“灯光明亮的路”上。嗯，我觉得，
23:43
multi- aent systems where you have multiple agents trying different approaches and you can evaluate you have maybe another model or another agent that's evaluating which ones of those seem promising is another way to kind of in some sense search the path of pos search the space of possible solutions and stick to the ones that seem most promising and discard the ones that that didn't seem to work or maybe that went off the rails. or whatever. Um, and that's a very very useful general technique is you know inference time compute to perform search over plausible ways of solving the problem that can get much much higher performance or much more reliability in longunning agent flows.
采用多智能体系统，让多个智能体尝试不同的方法，然后你可以评估——也许有另一个模型或另一个智能体来评估其中哪些看起来有希望——这也是一种在某种意义上搜索可能解法空间的方式，坚持那些看起来最有希望的，丢掉那些行不通的，或者跑偏了之类的。嗯，这是一个非常非常有用的通用技巧，也就是用推理时的算力去搜索各种可能的解题路径，这能带来高得多的性能，或者在长时间运行的智能体流程中带来高得多的可靠性。
24:30
>> How are some ways you implemented this particular workflow for your agents internally? Yeah, I mean we have uh you know harnesses and then we have a whole set of skills uh particularly in the internal Google development environment. We have skills so that the agents can know how to use lots of our internal tooling for coding or for code reviews or for you know measuring performance or you know fetching log files. And um those are just skills that you can add to make the base model more capable even though it hasn't necessarily been trained on exactly the way that you know Google internal uh engineers would fetch log files from our you know proprietary system with the right kind of skill uh definition you can actually get it to work >> uh and that that improves the usefulness of the agents. Now let's talk about uh where startups can can win. This section is one that I personally care a lot about because also everyone here in this room needs to decide what to build in the future of your future founder. So
>> 你们内部是通过哪些方式为自己的智能体实现这套工作流的？是啊，我们有脚手架，然后我们有一整套 skills，尤其是在 Google 内部的开发环境里。我们有 skills，让智能体知道如何使用我们大量的内部工具，比如写代码、做代码评审，或者测量性能、抓取日志文件。而这些就是你可以加上去的 skills，让基础模型变得更有能力，即使它并不一定被专门训练过Google 内部工程师是怎么从我们的专有系统里抓取日志文件的——只要有合适的 skill定义，你其实就能让它跑起来。>> 这也提升了智能体的实用性。那我们来聊聊创业公司能在哪里取胜。这一部分我个人非常关心，因为在座的各位都需要决定未来要做什么，作为未来的创业者。Google 的特点是，你们从处理器到产品
071% 法则：创业公司在哪能赢
25:37
the thing about Google is you co-design everything on the system from the processors to the products. um which are the layers that someone like Google would keep building and compounding being better and and where does a two three person team can still win? >> Yeah, I mean I think obviously Google and and our Gemini models and and our hardware infrastructure are really trying to build very general models that can do almost anything. But in in a lot of cases that means that we don't have a lot of attention on particular domains where perhaps a really well-designed surface that and maybe a model and set of skills or maybe a specialized model that uh isn't in sort of a general mix of of things that our models do well can actually have a significant advantage because you can build something delightful and you know really high accuracy. really high quality for a domain that you are really passionate about. And I think that's that's where you know the two or three people in a room uh building that that they're
对整个系统做协同设计。那么，哪些层是像 Google 这样的公司会持续构建、不断复利式变强的，而两三个人的小团队又能在哪里取胜？>> 是啊，我觉得显然 Google、我们的 Gemini 模型以及我们的硬件基础设施，都在努力构建能做几乎所有事情的非常通用的模型。但在很多情况下这也意味着我们不会在某些特定领域投入很多注意力，而在那些领域里，一个设计得非常好的交互界面，加上也许一个模型和一套skills，或者一个专门的模型——那种不在我们模型擅长的通用范畴里的——其实可以有显著的优势，因为你可以为一个你真正热爱的领域做出一个令人愉悦、准确率非常高、质量非常高的东西。我认为那就是两三个人在一个屋子里、做他们真正兴奋的事情能有优势的地方。嗯，但我也要提醒一句，通用模型确实在
26:49
really excited about can have an advantage. Um but I I would also caution that the general models are definitely getting better at a broader and broader range of things. So you have to figure out, you know, is that thing you're working on, is that going to be a durable thing or do you think the models uh at the forefront are going to get better at that in the next six months or 12 months or is it something they're not going to be able to do for a couple years or three years? And you know, you you want to weigh that as you're as you're deciding what to work on.
越来越广的范围上变得更强。所以你得想清楚，你正在做的这件事，它是不是一件能长期站得住的事，还是说你觉得最前沿的模型在未来六个月或十二个月内就会在这件事上变强，又或者这是它们两三年内都做不到的事？在决定要做什么的时候，你需要权衡这一点。
27:22
>> So let's uh dive deeper into this. So the general models of course you're going to keep working on and keep making them all better. And how should the audience reason about what are those areas that uh it doesn't I mean h how should founder think about things to pick on and work on. >> Yeah. I mean I mean the most important thing is to pick something you're super excited about and want to build and you think would be useful in the world, right? So if you do that um that that's you're already way ahead uh than if you wake up and you're like oh I don't really want to do this or whatever or you're going to build something that is actually not that useful to to the world or to to many people. Um so I think that's the number one selection criteria I try to apply for what problem should I work on next. Um, second, I think you want to look at what the current more general models can do in that problem domain, right? You can you can test them with like, are they able to do this thing very well? And if they're
>> 那我们再深入一点。通用模型你们当然会继续做，继续让它们全面变强。那么听众该如何判断哪些是它做不到的领域？我是说，创业者该如何思考该挑选并投入哪些方向？>> 是啊。我觉得最重要的是挑一件你特别兴奋、特别想做，并且你认为对世界有用的事，对吧？如果你做到了这一点，你就已经领先很多了，比起你早上醒来想“唉我其实不太想做这个”之类的，或者你要做的东西其实对世界、对很多人都没什么用。嗯，所以我认为这是我在选择下一个该做什么问题时会用的第一条标准。第二，我认为你要看看当前更通用的模型在那个问题领域能做到什么，对吧？你可以去测试它们，比如它们能不能把这件事做得很好？如果它们完全做不到，那大概是个好信号。如果它们能做一部分但做得不太好，那
28:26
completely failing, that's probably a good sign. If they're kind of able to do some of it but not very well, that's maybe not a great sign because that's a probably a a sign that the capability is starting to be present in those models and with more training data or larger scale models or or whatever it's likely to get better. So um you know look for something where the model succeeds 0% or 1% of the time not not 20%. >> How do you find those? I mean are those things effectively uh out of distribution from the training set and what exactly is the problem shape that fits that?
可能不是什么好信号，因为这大概说明这项能力已经开始在这些模型里出现了，随着更多训练数据、更大规模的模型等等，它很可能会变好。所以，你要找的是模型成功率为 0% 或 1% 的事情，而不是 20%。>> 那怎么找到这些？我是说，这些东西是不是本质上处于训练集分布之外？到底什么样的问题形态符合这一点？
29:04
>> Yeah, I mean I think uh sometimes it's uh a product that you build that might have access to particular kind of data that the underlying model might not the a general model. So it might be you're building something to help users organize all their own personal information and the model won't necessarily have access to that. And so there you can have a big advantage because all of a sudden your model has visibility or your product has visibility into important data. Um it could be some incredibly hard problem where if you get the right training data and you can train a more specific model than a general purpose one, you can actually do that in a very affordable way. you maybe it doesn't take that much compute to train a a niche model for this particular problem, but you can get something that's highly accurate. That can sometimes be a a really good uh building block for for solving a important problem that is maybe not handled very well by the general model.
>> 是啊，我想有时候是这样：你做的产品可能能接触到某种特定的数据，而底层模型、通用模型接触不到。比如说，你在做一个帮助用户整理他们全部个人信息的产品，而模型不一定能访问这些数据。那样你就能有很大的优势，因为突然之间你的模型、或者说你的产品，能看到重要的数据。嗯，也可能是某个极其困难的问题，如果你能拿到合适的训练数据，训练一个比通用模型更专门的模型，你其实能以非常低的成本做到这一点。也许为这个特定问题训练一个小众模型并不需要那么多算力，但你能得到一个准确率非常高的东西。这有时候会成为一个非常好的构建模块，用来解决通用模型可能处理得不太好的重要问题。
30:04
>> I think that's interesting. I think there are basically two paths. The first path is uh a little bit funny is uh you guys are organizing the world's information. >> Yeah, >> that's probably kind of well covered. >> Yeah. but organizing your personal information that's open which is funny. >> Yeah. >> And then the second path um you talked about more specialized models in certain domains. Can you tell us more about what are some of these domains? >> Yeah. I mean I think like if you look at uh my colleagues work on say alpha fold that was a very specific model for uh protein folding and it was highly successful um and was able to really handle that domain quite well so that all of a sudden you now have this amazing tool and model that can give you answers to questions about proteins and their structure um really effectively um but it's not a general model it's a very specific one and there are other I domains where that kind of approach can work really well. Uh maybe in material science or chip design or things like
>> 我觉得这很有意思。我觉得基本上有两条路。第一条路有点好笑，就是你们已经在整合全世界的信息了。>> 是的。>> 那块大概已经被覆盖得差不多了。>> 对。但整理你的个人信息这块还是开放的，这挺有意思的。>> 是的。>> 然后第二条路，你提到了某些领域里更专门的模型。你能多讲讲有哪些这样的领域吗？>> 好的。我觉得，比如你看我同事们做的 AlphaFold，那就是一个非常专门的模型，用于蛋白质折叠，它极为成功，能够把那个领域处理得相当好，以至于突然之间突然之间，你就有了这样一个了不起的工具和模型，能非常有效地回答关于蛋白质及其结构的问题。但它不是一个通用模型，而是一个非常专门的模型。还有其他一些领域，这种思路也能发挥得很好。比如材料科学、芯片设计之类的领域，它能让你借助
08规格说明与选题品味
31:07
that that uh will enable you to leverage the capabilities of a very accurate but but niche model uh to do things that are hard today. >> That's a good example. So if some of you find a problem that's similar shape like alpha fold could be a good problem to work on. Now let's assume you found a problem to work on. We're going to talk a bit about how do you become a AI native founder? How do you really become good at it? Uh you in the past said that managing a fleet of agents, it's like 50 or 100 agents is all about writing really good crisp design docs or specs.
一个非常精准但很小众的模型的能力，去做今天很难做到的事。>> 这是个好例子。所以如果你们中有人找到形态类似的问题，像 AlphaFold 这样的，可能就是个值得投入的好问题。那现在假设你已经找到了要做的问题。我们来聊聊，怎么才能成为一个 AI原生的创始人？怎么才能真正做得好？你以前说过，管理一支智能体大军，比如 50 个或 100 个智能体，关键就在于写出非常好、非常清晰的设计文档或规格说明。
31:45
>> And how do people get good at that? What what do those look like? Yeah, I mean I think uh it's you you'll have a lot more success when working with your virtual agents if you can clearly specify what it is you want. And the clearer you are on what it is you want, the more the agent will have sort of guidelines and sort of rules of, you know, an outline of what it is trying to accomplish. Um whereas if you don't specify very much stuff, the agent has to sort of infer what it is you meant. And in many cases, it might infer things that are different than what you imagined. So we've always told computer scientists from the very beginning that really it's really important to specify what it is, what's the software that you're writing is trying to accomplish before then going and writing it. And so now we actually have agent-based systems that can do the writing, but the importance of specifying what what it is you want has actually gone up because before you'd be handing it off to a very intelligent human who maybe has context
>> 那大家怎么才能把这件事做好？这些文档长什么样？是的，我觉得，你在跟虚拟智能体协作时，如果你能清楚地说明自己想要什么，成功率会高得多。你对自己想要什么说得越清楚，智能体就越能有一套指引和规则，也就是它要完成的目标的大致轮廓。而如果你什么都不怎么说明，智能体就得去猜你到底是什么意思。很多情况下，它猜出来的东西跟你设想的不一样。所以我们从一开始就一直告诉计算机科学家，非常重要的一点是，在动手写之前先明确说清楚：你要写的这个软件到底要完成什么。而现在我们确实有了能替我们写代码的智能体系统，但把你想要什么说清楚的重要性其实是上升了。因为以前你是把活交给一个非常聪明的人，他可能了解背景，
32:50
or can ask you follow-up questions. Um, and agents can sometimes do that, but I I think clear specifications is is a really good idea. Um, and to give you an example of a a a use of a coding agent that works extremely well is you can ask today's models to translate software from one computer language to another very effectively because in that case you actually have a incredibly detailed specification. You have the whole software that says what the system is supposed to do. And so if you have a Python implementation of something and you want a Go implementation of it, you know, that is something that the models seem incredibly capable at doing these days because you can it can sort of take all the tests that are in Python, make sure they pass in the Go version, translate the tests to Go, um you know, compare uh behavioral differences between the implementations until there aren't any um and be you know, highly effective because that spec is so clear.
也可以追问你问题。智能体有时候也能这么做，但我觉得清晰的规格说明真的很重要。举个编程智能体用得极好的例子：你可以让今天的模型把软件从一种编程语言翻译成另一种，效果非常好，因为在那种情况下你其实有一份极其详尽的规格说明。你有整套软件，它本身就说明了这个系统应该做什么。所以如果你有某个东西的 Python 实现，想要一个 Go 的实现，这就是现在的模型看起来极其擅长的事情，因为它可以把Python 里的所有测试拿过来，确保它们在 Go 版本里也能通过，把测试翻译成 Go，然后比较两个实现之间的行为差异，直到没有差异为止，而且效率非常高，因为那份规格说明太清晰了。
33:51
Hm. Now let's assume now every founder gets good at running hundreds of agents at the same time and all the code is written for them by the agents. What becomes the scarce skill? >> Yeah, I mean I think it's really having incredibly good taste in what you ask your agents to work on, right? That is the the crux of you know from my background uh a research problem. You know, a researcher can have all the tools and all the techniques, but often most of the battle is what problem are you gonna spend your time on? And if you pick the problem well and you succeed in in in solving it, that's way better than if you, you know, uh, delightfully execute a research investigation into a rather boring problem. And so that high level wisdom of what to work on, I think is incredibly important. And I think models are not necessarily going to be that good at it. So you're going to have people steering uh a lot of AI assisted computation in order to accomplish great things and more quickly. Um but that essence of of
嗯。那现在假设每个创始人都很擅长同时驱动上百个智能体，所有代码都由智能体替他们写好了。这时候什么会成为稀缺的能力？>> 是的，我认为关键在于你对「让智能体去做什么」有极好的品味，对吧？这就是从我的背景来说，一个研究问题的核心。研究者可以拥有所有的工具和技术，但往往胜负手在于：你要把时间花在哪个问题上。如果你选对了问题并且成功解决了它，那比你把一个相当无聊的问题的研究过程执行得再漂亮都要好得多。所以「该做什么」这种高层次的智慧，我觉得极其重要。而且我觉得模型未必会特别擅长这一点。所以会是人来掌舵，去引导大量 AI 辅助的计算，从而更快地完成伟大的事情。但你希望模型去做什么，这个本质
35:02
what it is you want your models to do is the the the key thing you should focus on. >> So let's talk a bit more about taste because it gets talked a lot about right now in this current era with agent coding. How do you exactly build taste and do that? I mean, yeah, that sounds so esoteric. How do you make it concrete? >> Yeah, I mean it it is a difficult thing. It's not like there's a measurable objective of of taste in a lot of cases. Um, I think some of it is from experience. You know, working on a lot of different problems in the past kind of teaches you about what kinds of problems might be interesting in the future or what kinds of things might be just barely possible by cobbling together these previous approaches and then some open problems you might have to work on in order to get to something kind of magical or or you know, highly useful. Um, another way you can get more experience for yourself is to just write down a bunch of things you think might be important in the next 12 months. And
才是你该聚焦的关键。>> 那我们再多聊聊品味，因为在现在这个智能体编程的时代，大家特别爱谈这个。你到底要怎么培养品味、怎么做到？我是说，这听起来太玄了。怎么把它落到实处？>> 是的，这确实是件难事。很多情况下，品味并没有一个可衡量的目标函数。我觉得其中一部分来自经验。过去做过很多不同的问题，会让你多少懂得未来哪类问题可能有意思，或者哪些事情把以前的方法拼一拼就刚好可能做成，以及为了做出某种近乎神奇、或者说非常有用的东西，你还得攻克哪些开放问题。另一种给自己积累经验的方式是，把你认为未来 12 个月里可能重要的一堆事情写下来。也许你会挑其中一件去做，
09思想实验：不可靠晶体管与 MapReduce
36:10
maybe you pick one of them to work on, but go back and evaluate in 12 months of these other things, which ones actually seemed important or which ones did other people in the world go out and and create and which ones did they did not seem to do yet. um that can give you a lot more samples for your own sort of taste creation uh capability. Um and and that's an important skill to have. >> I think a third way we were talking earlier was doing very crazy thought experiments. >> Oh yeah, that's another good way. I mean I think uh sometimes it's good to not take as a given things that most people seem to take as a as a given. Um so I was doing a crazy thought experiment with some colleagues the other day about you know for 60 years the whole silicon uh chip design industry uh design and fabrication industry have you know done tremendous work to make smaller and smaller scale transistors that are uh very low error rate right like because what what the assumption that we want is that every chip we
但 12 个月后回头去评估其他那些事：哪些真的显得重要，哪些被世界上的其他人做出来了，哪些看起来还没人做。这样能给你的品味养成能力提供多得多的样本。这是一项很重要的技能。>> 我们之前聊到的第三种方式，是做非常疯狂的思想实验。>> 对，那也是个好办法。我觉得有时候，把大多数人视为理所当然的事情别当成理所当然，是很有价值的。前几天我就和几位同事做了一个疯狂的思想实验，过去 60 年里，整个硅芯片设计与制造行业做了大量的工作，把晶���管做得越来越小，而且错误率非常低。因为我们的假设是：同一设计制造出来的每一块芯片，都应该和其他芯片完全一样。
37:25
manufacture of the same design should be identical to every other chip. >> You don't want any bits to flip. Everything >> no bits should flip. There's all kinds of things you there's all kinds of error margins built into you know memories have ECC memory these days. um you know at the at the macro scale we don't make that assumption when we're building large scale distributed systems right we we build reliable large scale distributed file systems out of unreliable parts right like individual discs can fail but your data should be safe and so we have mechanisms at a higher level to enable us to have um you know three copies of the data on three different machines and three different racks so that if any rack switch or individual ual machine or disk fails, you still have your data. We have read Solomon encoding techniques. Um, but we don't seem to do this at a really extreme level in the uh sort of transistor level scale of of the technology we're working on. So what would h basically a interesting thought
>> 你不希望有任何比特翻转。所有东西 >> 不应该有比特翻转。当然也有各种误差余量设计，比如现在的内存有 ECC 内存。但在宏观尺度上，我们建大规模分布式系统时并不做这种假设，对吧？我们用不可靠的部件搭出可靠的大规模分布式文件系统，单块磁盘可以坏，但你的数据必须是安全的。所以我们在更高层次上有各种机制，让我们能把数据的三份副本放在三台不同机器、三个不同机架上，这样任何一个机架交换机、单台机器或磁盘挂了，你的数据还在。我们还有里德-所罗门编码之类的技术。但我们似乎没有在晶体管这个层级的尺度上，把这套思路做到很极端的程度。所以一个有意思的思想实验就是：如果你试图用那种一天可能出 20 次错的晶体管
38:30
experiment is what would happen if you tried to build a system out of transistors that might have you know 20 errors per day. >> Oh my god. rather than one every million years, right? That would be a very different design point and might be might enable you to do really interesting things in the fabrication side of things. You have very different kind of design methodologies because if you want to get a signal from here to there, you and you have these super unreliable transistors. You might have very different ways of signaling. You might send it along multiple redundant paths uh in order to make sure that it gets along one of them. Um, and I think that would be a pretty interesting set of thought experiments. I'm not saying we should go do this, but you know, that's the kind of thing where you do want to, you know, occasionally question assumptions. Now, oftentimes these thought experiments don't work out because there are very good reasons that, you know, for the last 50 years, we've done this thing this way and not
来搭系统，会发生什么？>> 天哪。而不是一百万年才出一次错，对吧？那会是一个截然不同的设计点，也许能让你在制造这一端做出真正有意思的事情。你会有非常不同的设计方法论，因为如果你要把一个信号从这里传到那里，而你手上的晶体管极其不可靠，你可能就得用很不一样的信号传递方式。你可能会沿多条冗余路径同时发送，以确保它至少能从其中一条到达。我觉得那会是一组挺有意思的思想实验。我不是说我们真该去做这件事，但这就是那种你确实应该偶尔去质疑一下假设的例子。当然，这类思想实验很多时候是不成立的，因为过去 50 年我们之所以这么做而不是那么做，是有非常充分的理由的。但每隔一段时间重新审视一下总是好的。
39:28
that way. But it it's good to kind of revisit those every so often. >> That is so wild. Well, I mean, it's starting to rhyme a lot with neuromorphic computing or the human brain and and how nature works. >> I mean, exactly like signals in our brain are not especially reliable from getting one place to another. And so, I think in brains when there are really important things you need to get from one place to another, there are multiple pathways that that enable you to sort of do that. >> What is uh in I mean, you have such an impressive career. What is one of these crazy assumptions that you threw out of the window that actually built a consequential system in the past?
>> 太疯狂了。我是说，这跟类脑计算、跟人脑、跟大自然的运作方式开始有很多相通之处了。>> 没错，我们大脑里的信号从一处传到另一处其实并不特别可靠。所以我觉得在大脑里，当有非常重要的信息需要从一处传到另一处时，会有多条通路来帮你完成这件事。>> 那么，你的职业生涯这么令人钦佩，有没有哪一个你曾经抛弃掉的疯狂假设，最后真的造出了一个影响深远的系统？
40:09
>> Yeah, I mean I guess uh >> that worked out actually. >> Yeah, I mean I think uh well TPUs is a good example like being able to specialize hardware for a very niche >> problem domain before that problem domain seemed as important as it is today uh is one thought experiment. Um you know I think the the origin of map produce is another good example. So we had worked the you know my Sanjay and myself and a number of other colleagues had worked on various iterations of the crawling and indexing system at Google and you know we'd sort of written lots of hand parallelized code with lots of checkpointing to make sure it would be robust and reliable if it was running on a 100 computers or a thousand computers and some of those died. Um, but that code tended to be intermixed with the actually relatively simple thing you often were trying to do like I just want to like look at all the contents of all the web pages and then compute on the side a mapping from URL to you know what language is this page
>> 嗯，我想…… >> 而且真的成功了的。>> 是的，我觉得 TPU 就是个好例子——在某个问题领域还没有像今天这么重要之前，就能为这个非常小众的领域做专用硬件，这算是一个思想实验。另外我觉得 MapReduce 的由来也是个好例子。当时我、Sanjay 还有其他一些同事，做过 Google 抓取和索引系统的好几个版本迭代。我们写了大量手工并行化的代码，加上大量检查点，以确保在 100 台或 1000 台机器上跑、其中一些机器挂掉时，系统仍然稳健可靠。但那些代码往往和你真正想做的、其实相当简单的事情混在一起，比如我只是想看看所有网页的内容，然后顺带算出一个从 URL到「这个页面的正文是什么语言」的映射。而这些逻辑会被大量
41:16
in the text of this page. Um, and it would get obscured by all this kind of other code for parallelization and reliability. And so we sort of remembered our training in functional languages and realized we could squint at those problems and developed this map produce abstraction that you could have above this implementation and then below the implementation you could put all the checkpointing and reliability mechanisms into that lower level library that everything could then build on. And so that became a hugely successful way of of dealing with very large scale computations at Google in a robust and reliable way. From that thought experiment of like well if we squint at it could we find lots of problems that fit into this abstraction.
用于并行化和可靠性的其他代码淹没。于是我们想起了自己在函数式语言方面的训练，意识到可以换个角度去看这些问题，就发展出了 MapReduce 这个抽象：在实现之上是这层抽象，而在实现之下，你可以把所有检查点和可靠性机制放进那个底层库里，让所有东西都构建在它之上。于是这就成了 Google 处理超大规模计算的一种极其成功的方式，既稳健又可靠。而这一切都来自那个思想实验：如果换个角度看，我们能不能找到很多适合这个抽象的问题。
10自动化科学方法与快速评估器
42:03
>> That's impressive. So this thought experiment led you to create map reduce. >> Yeah. Awesome. >> Now let's go back to you talked a bit about um about your interest right now working on a lot of customized hardware. So right now alpha chip >> lays out chips. Now you also got alpha evolve that proposes solutions, >> evaluates them and keeps all the ones that work. Seems like you're starting to build all these system that can compound and build AI that builds AI. >> Yeah. I mean I think more generally there's a there's this sort of the foundation of the scientific method of you propose an experiment you implement what you need to run the experiment and you evaluate the experiment and then you get results from that and I think there are more and more problems that are now possible to implement where that whole loop of running you know not just a few experiments but running many many experiments because you're able to automate that loop and make the latency of that loop extremely low is going to
>> 太厉害了。所以这个思想实验让你们创造了 MapReduce。>> 是的。太棒了。>> 我们回过头来。你刚才提到你现在对做大量定制化硬件很感兴趣。现在有 AlphaChip >> 来做芯片布局。你们还有 AlphaEvolve，它会提出解决方案、>> 对方案做评估，并保留所有有效的方案。看起来你们正在构建这样一整套能够复利叠加、用 AI 造 AI 的系统。>> 是的。我觉得更普遍地看，这其实是科学方法的基础：你提出一个实验，实现运行这个实验所需的东西，评估这个实验，然后得到结果。我觉得越来越多的问题现在可以这样来做了——整个循环不只是跑几个实验，而是跑非常非常多的实验，因为你能把这个循环自动化，并把循环的延迟压到极低，这会变得
43:08
be really really important. It's going to enable us to tackle you know lots of different problem domains in science and engineering and machine learning uh model design itself and also in engineering tasks like designing chips. And so if you can actually do those things in an automated way and have some orchestration framework that can take very high level objectives and break them down into subpros and each of those subpros can be one of these automated loop that is exploring the best way to solve that sub problem and then a orchestration framework that can put together subpros solutions into a you know the overall solution for the higher level problem that's going to be really impactful and it's really really important and I think it'll enable us to do you know accelerate machine learning progress it'll enable us to accelerate science and enable us to accelerate engineering and I think that's that's going to be amazing >> that sounds awesome I mean it sounds like a lot of fields basically where you
非常非常重要。它会让我们能攻克科学、工程、机器学习模型设计本身，以及像芯片设计这样的工程任务中，许许多多不同的问题领域。所以如果你真能用自动化的方式做这些事，并且有某种编排框架，能接收非常高层次的目标，把它们拆解成子问题，而每个子问题都可以交给这样一个自动化循环去探索最佳解法，然后再由编排框架把子问题的解拼装成更高层问题的整体解决方案——那将会非常有影响力，也非常非常重要。我觉得它能让我们加速机器学习的进展，能让我们加速科学，也能让我们加速工程，我觉得那会非常了不起。 >> 听起来太棒了。我是说，听起来很多领域都是这样：只要你能有很好的评估器，或者接近那些可以被
44:12
can have very good evaluators and maybe adjacent to basically things that can be formally verified right those are ripe for AI systems that can self-improve Yeah, I think in a lot of cases sometimes your evaluators need to be made much faster. Mhm. >> So as an example, my colleagues did some work maybe a decade ago on um some uh problems in quantum chemistry where you're trying to understand the properties of a particular molecule and you can you know generate some molecule configuration and then you want to understand what properties it has. And so you can run a very computationally intensive density functional theory simulator which is something that might take like a a night of computation to tell you the answer for one thing. Um but what my colleagues did was take a bunch of output from those simulation runs the input molecule configurations and the outputs of the the expensive simulator and then use it to train a neural approximation to the simulator. So this is now a validation device, but instead of it taking a
形式化验证的东西，那这些领域就非常适合能自我改进的 AI 系统。是的，我觉得很多情况下，你的评估器需要被做得快得多。嗯。>> 举个例子，大概十年前我的同事们做过一些量子化学方面的工作，你要理解某个特定分子的性质，你可以生成某种分子构型，然后想知道它有什么性质。于是你可以跑一个计算量非常大的密度泛函理论模拟器，可能要算一整晚才能告诉你一个结果。但我的同事们做的是，拿一批那些模拟运行的输出——输入的分子构型，以及那个昂贵模拟器的输出——然后用这些数据去训练一个神经网络近似模型，模拟器。所以这现在是一个验证工具，但它不需要花一整晚，他们做出来的东西快了 30 万倍。
45:22
night, they made something that was 300,000 times faster. >> Wow. >> And nearly as accurate as running the full scale simulator. So now that completely changes how you would do science, right? Because now you have 10 million things to screen. you know, you could do that while you go to lunch rather than it being a six-month endeavor where you could try to scrape together enough compute to to run all these simulations. And I think there's a lot of room in a lot of domains for much faster validation models, possibly learned valu validation models that can uh you know get you a a approximation to the true answer much much more rapidly.
>> 哇。>> 而且几乎和跑完整规模的模拟器一样准确。所以这就彻底改变了你做科学研究的方式，对吧？因为现在你有 1000万个东西要筛选，你去吃个午饭的工夫就能搞定，而不是变成一个长达六个月的大工程，还得想办法东拼西凑足够的算力来跑完所有这些模拟。我觉得在很多领域里，都有很大的空间去做更快的验证模型，可能是学习出来的验证模型，能让你更快得到接近真实答案的近似结果。
46:04
And that changes how those experimental loops can be thought of and how quickly you can go around those loops. >> What are some of the spaces and problems that you're super excited that this super sped up scientific method is going to solve or achieve? What particular problems or spaces? >> Yeah, I mean I think uh well clearly machine learning itself is one, right? So can we have a model that is able to recursively self-improve itself by running lots of experiments and you know if you think about how models are improved today in large research teams you know what usually happens is people think of some ideas they run a bunch of smallcale experiments they see if those small scale experiments worked out well if so they take the most promising ones of those they try them at larger scale and that gets then evaluated and then the results get integr ated together into you know a new recipe for your model. Um but I think there's no uh you know real impediment to making that be a much more automated loop where the model itself
这就改变了我们思考那些实验循环的方式，以及你能多快地跑完这些循环。>> 有哪些领域和问题是你特别期待这种大幅加速的科学方法能够解决或实现的？具体是哪些问题或领域？>> 是的，我是说，我觉得呃，很明显机器学习本身就是其中之一，对吧？我们能不能有一个模型，能够通过跑大量实验来递归地自我改进？你想想今天在大型研究团队里模型是怎么被改进的，通常的情况是，大家想出一些点子，跑一堆小规模实验，看看这些小规模实验效果好不好，如果好的话，就挑出其中最有希望的几个，在更大规模上试，然后进行评估，接着把结果整合到一起，形成你模型的新配方。嗯，但我觉得没有什么呃真正的障碍阻止我们把这变成一个自动化得多的循环——由模型自己决定去探索，或者说
11蒸馏被拒与职业路径选择
47:15
decides it's going to explore or maybe with a nudge from some people uh at the various highest level like oh why don't you try some new ideas around model architectures that incorporate this and then it will go run lots of experiments uh see which ones work and then those will get incorporated at a much more rapid rate and uh you know effectively you want to optimize you know your discoveries per unit of compute input. >> Very cool. >> Yeah. >> Now going back to the room as all of you will become at some point founders or start your careers you will probably collect lots of rejections. That will happen. Uh it has happened to you too Jeff. I mean there's a story that in 2014 you with Jeff Hinton and Oral Fin wrote a paper on distillation >> which has to do with taking a big teacher model to train a much smaller and more efficient model that's a lot cheaper to compute less model parameters and it has become a trick that everyone is using right now in industry. Yeah.
在最高层面上得到人的一点提示，比如说，你不如试试围绕模型架构、结合这个东西的一些新想法，然后它就会去跑大量实验，呃看看哪些管用，然后这些就会以快得多的速度被整合进来。呃，你知道，本质上你想优化的是每单位算力投入所带来的发现量。>> 非常酷。>> 是的。>> 那我们回到在座各位——你们中的一些人以后会成为创业者，或者刚开始职业生涯，你们大概会收到很多拒信。这是会发生的。呃，你也经历过，Jeff。我是说，有个故事是，2014 年你和 Geoff Hinton、Oriol Vinyals 一起写了一篇关于蒸馏（distillation）的论文 >> 就是用一个大的教师模型去训练一个小得多、更高效的模型，计算成本低很多，模型参数也更少，而它现在已经成了业界人人都在用的技巧。是的。
48:30
>> And the thing is this paper got rejected at Europe. >> Yeah. I mean Yeah. I mean I think I don't fault the program committee because you know a lot of times a paper gets three reviews and someone will look at one of the reviewers will look at it and in this case they said oh it's unlikely to have significant impact. >> Unlikely to have significant impact. But you know I think you know when we wrote the paper we actually saw this was a super important problem because we knew making cheaper highly capable models from larger scale models was something we desperately wanted to do because we wanted to serve models to more and more people in many different domains like speech or vision. Um but you know sometimes the reviewer maybe didn't have that that experience because maybe they're not thinking about you know largecale AI services and are thinking about you know is this a fundamental advance um so so you know it gets rejected every so often that's fine we put it on archive people read it people
>> 问题是，这篇论文被 NeurIPS 拒了。>> 是的。我是说，是的。我觉得我不怪程序委员会，因为你知道，很多时候一篇论文会拿到三份评审，有人会看一眼——其中一位审稿人看了之后，这次他们说，哦，这不太可能产生显著影响。>> 不太可能产生显著影响。但你知道，我觉得我们写这篇论文的时候，其实是看到这是一个超级重要的问题，因为我们知道，从大规模模型里做出更便宜但能力很强的模型，是我们非常迫切想做的事，因为我们想把模型服务提供给越来越多的人，在很多不同领域，比如语音或视觉。嗯，但你知道，有时候审稿人可能没有那样的经历，因为也许他们没在想呃大规模 AI 服务这件事，而是在想呃这是不是一项根本性的突破。嗯，所以你知道，隔三差五被拒一次，没关系，我们把它放到 arXiv 上，大家读了，大家用了，都挺好。呃，你知道，我们确实在做 Flash 模型时用到了它，比如
49:31
use it it's all good uh and you know we do use it in making our flash models for example from our larger scale pro model that's partly why our flash models for example in Gemini are so capable uh relative to their size and and speed. >> They're some of the best in the benchmark for their model size class. Yeah. Just impressive. And I think part of the lesson is that even if you get rejected, keep going. >> Yeah. That's that's the lesson I would distill from that. [laughter] >> Um no, I think the fun thing is that you basically join when you when you join Google as a 20 person startup back in 1999.
从我们更大规模的 Pro 模型蒸馏出来，这也是为什么我们 Gemini 里的 Flash 模型相对于它们的体量和速度来说，能力这么强的部分原因。>> 在同等模型规模级别的基准测试里，它们属于最好的一批。是的，真的很厉害。我觉得其中一部分教训是，就算被拒了，也要坚持下去。>> 是的。这就是我会从中"蒸馏"出来的教训。[笑声] >> 嗯，不过我觉得有意思的是，你基本上是在 Google 还是一家 20 人的创业公司时就加入了，那是在1999 年。
50:09
Now, if you were to take the young Jeff Dean from way back then to teleransport him to now today. >> Yeah. >> In this era with your skills. >> I'm feeling so vigorous and and young now. >> Um what would you do? Do you join a frontier lab, start a company? I don't know what what would you do? The c the Jeff theme today 25-year-old Jeff theme. >> Yeah. I mean, it's always hard to say and it's a very personal choice of what it is you want to spend your time on. Um, to me, some of the most important questions are, are you going to work on something you really care about, will you're working on that? And if you're able to make progress on it with a bunch of colleagues you like working with uh if you're able to make collectively solve it or make progress on it, will that make a difference in the world in some positive way, right? Like will you suddenly be able to do something and offer that service to you know partically help biochemists or something or maybe it's a broader thing. It'll help programmers or it will help all
那么，如果把当年那个年轻的 Jeff Dean 瞬移到今天。>> 是的。>> 在这个时代，带着你的技能。>> 我现在感觉浑身是劲、特别年轻。>> 嗯，你会做什么？你会加入一个前沿实验室，还是自己创业？我不知道，你会怎么做？今天这个 25 岁的 Jeff Dean。>> 是的。我是说，这always很难说，而且这是一个非常个人化的选择，取决于你想把时间花在什么上。嗯，对我来说，有些最重要的问题之一是，你会去做一件你真正在乎的事情吗？你会投入其中吗？如果你能和一群你喜欢共事的同事一起在这件事上取得进展，如果你们能共同解决它或者推进它，那这会不会在某种正面的意义上改变世界？比如说，你会不会突然能做成某件事，并把这项服务提供出去，比如切实地帮助到生物化学家，或者也许是更广泛的事情。它能帮到程序员，或者能帮到所有消费者，帮到互联网上的人，等等。嗯，你应该努力去做的，是在世界上产生
51:24
consumers. uh on the internet or or other things. Um what you you know what you should strive to do is to have impact in the world that is positive and to work with people you enjoy working with and to you know uh work hard and and do your best. Um so in terms of say the particular trade-off you offered joining a frontier lab versus say starting a company with just one or two or three of you you and your close friends. Um, I think those are different experiences, right? In a in a large established organization, you have some structure. You have lots and lots of amazing colleagues who know lots of things you don't. Um, you have lots of interesting problems that uh you can work on and h you already have a platform for impact by your work, you know, influencing lots and lots of people in the world already. Um and then as a very small startup, you know, you have to have something you're passionate about and there's a lot of risk in taking on, you know, working on that particular problem in a
正面的影响，和你喜欢共事的人一起工作，然后努力工作，尽你所能。嗯，所以说到你提出的那个具体权衡——加入一个前沿实验室，还是和一两个或三个你和你的好朋友一起创业。嗯，我觉得这是两种不同的体验，对吧？在一个大的、成熟的组织里，你有一定的架构。你有非常非常多了不起的同事，他们懂很多你不懂的东西。嗯，你有很多有意思的问题可以去做，而且你的工作本身已经有了一个产生影响的平台，已经能影响到世界上非常非常多的人。嗯，而作为一家很小的初创公司，你必须得有一件你充满热情的事情，而且要承担很大的风险，去用某种方式攻克那个特定的问题，让自己能成功，能把这份
52:36
way that uh you're going to succeed and you're going to grow a, you know, an endeavor in order to do that. But that can also be incredibly rewarding, I would imagine. So I I think um you know it's really up to personal taste but but at the very least regardless of what path you take ask yourself if I work on this problem and the best possible outcome happens you know will the world be a lot better in some way or will the world go eh that's kind of cool but whatever. >> Uh that's not the kind of thing you should spend your time on.
事业做大做起来。但我想，那同样可能是极其有成就感的。所以我觉得，嗯，这真的取决于个人的偏好。但至少，不管你选哪条路，都问问自己：如果我做这个问题，而且发生了最好的结果，那世界会不会在某种意义上变得好很多？还是说世界只会说，嗯，挺酷的，但也就那样。>> 呃，那就不是你该把时间花在上面的事情。
12组队原则与留给后来者的问题
53:10
Now let's talk a bit about more about that second path of working with people that you really like in a small team. You've been able to be an incredible mentor and manager to many many engineers and you've been able to build huge systems and what are some some of the lessons for everyone here on how to get the most and how to work with smart people or find smart people? Yeah, I mean, you always want to find people who have really good skills in some some area that's needed in, you know, a team you're trying to form, whether that's inside a company or uh starting a company. Um, but you also want to find people that are people you delight being around, right? because you're going to spend a lot of time around people working on really hard problems and you want people who are low ego that are team players that you know have complimentary skills to your own perhaps um I always find working in a small team where people know things that I don't know and where maybe I have some skills that other people don't have as much of
现在我们再多聊聊第二条路——和你真正喜欢的人在一个小团队里共事。你一直是许许多多工程师非常出色的导师和管理者，你也构建过庞大的系统。对于在座的各位，关于如何收获最多、如何与聪明的人共事或者找到聪明的人，你有什么经验？是的，我是说，你总是希望找到在某个领域有非常好的技能的人，而那正是你想组建的团队所需要的，不管是在公司内部还是创业。嗯，但你也希望找到那些和他们相处让你感到愉快的人，对吧？因为你会花大量时间和这些人一起攻克真正困难的问题，你希望这些人自我意识不强、有团队精神，并且可能拥有与你互补的技能。嗯，我一直觉得在小团队里工作在那里别人知道我不知道的东西，而我可能有一些别人没那么擅长的技能
54:22
you know is super fun because you're collectively building something or working on something that none of you could maybe do individually. ually, but in the process of working on that, you actually gain a lot of new knowledge and new skills uh for yourself and so do they. And you you kind of want to view your engineering or research career as you have an amazing tool belt of techniques. And you always want to be adding new tools to that tool belt because you never know when you might come across a problem where you need these four specialized tools rather than these three. And adding more tools makes it more likely that the problems you you encounter in the future will be solvable by you.
这真的特别有意思，因为你们是在共同构建某个东西，或者共同做一件你们任何一个人单独都做不成的事。但是在做这件事的过程中，你自己实际上会获得很多新知识和新技能，他们也一样。所以你应该这样看待自己的工程或研究生涯：你有一条装满各种技术的超棒工具腰带。而你要一直往这条工具腰带上添加新工具，因为你永远不知道什么时候会遇到一个问题，需要的是这四种专门的工具，而不是那三种。工具越多，你未来遇到的问题就越有可能被你解决。
55:07
>> Now, one last thing. I'm pretty sure someone in this room or multiple people will eventually build something as consequential as you've done with map reduce, TPU, distillation, etc., etc. What problem do you hope they would be working on? Oh yeah. I mean I I think there's a lot of interesting problems in the world and I'll just rattle off a few. This is not exhaustive because the world is a very big place and full of problems. You know I'm particularly excited about new approaches to hardware. You know we that thought experiment there was kind of you know a you know a indication of that or much more efficient inference hardware. You know, I think there are radically different kinds of algorithms for machine learning that might be much much more data efficient than the approaches we're using today. If you think about our large scale models today, they probably see a thousand times as much data as a human does by the age of 18.
>> 最后一个问题。我很确定，这个房间里的某个人、或者好几个人，最终会做出跟你做的 MapReduce、TPU、蒸馏等等一样有影响力的东西。你希望他们去解决什么问题呢？哦，是啊。我是说，我觉得这世界上有很多有意思的问题，我就随口说几个。这并不是全部，因为世界非常大，充满了各种问题。你知道，我特别期待硬件方面的新思路。我们刚才那个思想实验其实就有点这个意思，或者说更高效的推理硬件。我觉得可能存在完全不同类型的机器学习算法，它们的数据效率可能远远高于我们今天用的这些方法。想想我们今天的大规模模型，它们看过的数据可能是一个人到 18 岁时所见数据量的一千倍。
56:09
Yet, the human by the age of 18 is better in a lot of things and, you know, on par uh with those frontier models that have seen way more data. So could you come up with much more data efficient systems that can learn continuously learn from their own actions? Uh continual learning is a really interesting thing. I think multi- aent interactions is an interesting thing. Um you know I think you know creating ways of having better discourse among people in the world uh could be interesting. Are there ways to have much more civil conversations and you know helping people meet other people are all over the world that they should know based on their interests. You know these are kind of interesting things. I think there there's lots of cool things in the world and we should all go and strive to make even cooler things occur.
然而这个人到 18 岁时，在很多事情上都做得更好，而且跟那些看过多得多数据的前沿模型不相上下。那么，你能不能设计出数据效率高得多的系统，能够从自己的行动中持续学习？呃，持续学习是个非常有意思的方向。我觉得多智能体交互也是个有意思的方向。嗯，我觉得，创造一些方式让世界上的人们能有更好的对话交流，可能也很有意思。有没有办法让对话更加文明？还有帮助人们认识世界各地那些基于兴趣本该认识的人。你知道，这些都是挺有意思的事情。我觉得这世界上有很多很酷的东西，我们都应该去努力让更酷的东西出现。
56:59
>> That sounds wonderful. Thank you so much Jeff Dane. That's all we have today. >> Appreciate it. >> Thank you all.
>> 听起来太棒了。非常感谢你，Jeff Dean。今天就到这里。>> 谢谢，很荣幸。>> 谢谢大家。
视频总结 · 一句话概括与核心要点
一句话概括
Jeff Dean 认为 AI 的下一阶段是"自动化实验循环"与"专用推理硬件"，而创业者应挑选通用模型成功率只有 0%~1%（而非 20%）的问题、写清晰的规格说明，并把"选题的品味"当作最稀缺的技能。
核心要点
- "初级工程师"预测基本应验，且被低估了：2025 年 5 月他预测 AI 达到初级工程师水平，一年后他认为已"相当准确"。低估之处在于模型完成复杂任务的能力增长快于预期，且 agent 系统在编程之外的领域也开始发光。2027 年的预测：ML 系统自我改进——自动把问题拆分为子问题、跑紧密的自动实验循环、整合结果，适用于一切有可测量目标的科学与工程领域。
- 当下的"索引装进内存"时刻是低延迟、低能耗的专用推理硬件：2001 年他与 Sanjay 算出搜索索引可全部放进 RAM，几天内上线全新版本。今天的类比是：推理才是让 agent 服务更多人的关键，延迟至关重要；比 GPU/TPU 更专用的硬件可以把延迟降低 50 倍，"想象一下不用等待能做什么"。他个人正在思考的方向：最小化数据搬运、极低精度运算、只支持一两种精度而非一大堆。
- 多数人尚未内化的事实：agent 可以连续运行数天到数周：不只是一两小时，在特定领域配合强模型，可以执行如"用另一种语言（更安全或更高性能）完整重写一套软件"这样的任务。
- TPU 源于一道餐巾纸算术：2013 年深度学习语音模型把错误率砍半（相当于 20 年的语音识别进步压缩到几个月），但算力昂贵。若每个用户每天用语音 3 分钟，Google 需把服务器数量翻倍。于是造了只做"低精度稠密线性代数"的专用芯片：能效比当时 CPU/GPU 高 30~80 倍，延迟低 20~30 倍。刻意做成"通用线性代数"而非过度特化，因此后来的 Transformer 也能受益。
- 能量是新的度量单位，很多"模型问题"其实是数据 IO 问题：一次乘法约 1 皮焦，把数据从 HBM 搬进处理器的能耗是其 1000 倍。正因这一差距才需要 batching 来摊销数据搬运——batch 训练、epoch 等设计本质是系统/IO 约束，而非模型本身的需求；而 batching 对低延迟推理并不友好。AI 时代的"每个工程师应知道的数字"应包括：加速器主存到片上内存到乘法器的带宽、单次乘法能耗、芯片间互联带宽、从 500 芯片扩到 1 万芯片时的网络带宽衰减。
- 上下文工程是人人可做的杠杆，skill 能把"Jeff Dean 的方法"交给模型：训练模型需要巨额资源，但上下文工程只需一个 API。他与 Sanjay 几周前写了一个 skill，把"跑微基准→修改代码→重跑→测缓存占用→迭代"的性能优化流程教给模型，效果不错。他们几个月前公开发布了一份约 30 页的《Performance Hints》文档，有人将其摘要喂给模型后，模型对代码性能的推理能力明显提升——免费可用。
- Agent 在第 30~50 步跑偏的原因是脱离训练分布：一旦任务偏离模型熟悉的范围，性能像所有 ML 模型一样急剧退化。对策：用 skill 和提示把模型留在"灯光明亮的路径"上；多 agent 尝试不同路径，再由另一个模型评估、保留有希望的、丢弃跑偏的——本质是用推理时算力做解空间搜索。Google 内部为编码、代码审查、性能测量、拉取日志等专有工具都写了 skill。
- 创业选题的 1% 法则：先测通用模型在该领域的表现——完全失败（成功率 0%~1%）是好信号；"能做一些但做不好"（约 20%）是坏信号，说明能力已萌芽，更多数据或更大规模就会补上。两类机会：(1) 产品能接触到通用模型拿不到的数据（如用户个人信息——"组织世界的信息"已被覆盖，"组织你的个人信息"仍开放）；(2) 像 AlphaFold 那样用特定训练数据低成本训练高精度的窄域模型，如材料科学、芯片设计。同时要衡量：前沿模型是否会在 6~12 个月内追上。
- 管理上百 agent 的核心是清晰的规格说明，最稀缺的技能是品味：规格越清晰，agent 需要猜测的越少；把 Python 翻译成 Go 之所以效果极佳，正因为原有代码和测试构成了极详尽的规格。当代码都由 agent 写时，稀缺的是"决定让 agent 做什么"的品味。培养方法：积累经验；写下你认为未来 12 个月重要的事，一年后回看哪些真的重要、哪些被别人做了；做疯狂的思想实验，不把"理所当然"当作前提。
- 自动化科学方法的瓶颈常在评估器速度：同事十年前用 DFT 模拟器（一次需一整晚）的输入输出训练神经近似器，速度快 30 万倍且精度接近，让"筛选 1000 万个分子"从半年工程变成午饭时间。目标函数应是"单位算力的发现数"。AlphaChip、AlphaEvolve 都是这一"提出→实现→评估→保留"循环的实例。
结论与值得注意的细节
- 思想实验案例：60 年来芯片业追求每百万年出错一次的晶体管；若换成"每天出错 20 次"的晶体管会怎样？这会带来完全不同的制造与设计方法（多冗余路径传信号），类似大脑中重要信号走多条通路，也类似分布式文件系统用不可靠磁盘构建可靠存储。他强调"不是说该去做"，但定期质疑前提是有价值的。MapReduce 正是这样诞生的：把并行、检查点、容错代码下沉到底层库，用函数式语言的视角"眯着眼看"大量问题后抽象出来。
- 蒸馏论文曾被 NeurIPS 拒稿，审稿意见是"不太可能有显著影响"；如今它是 Gemini Flash 模型相对其体积如此强大的关键原因之一。教训：被拒就放到 arXiv 上，继续做。
- 数据效率是巨大的开放问题：大模型看到的数据约是人类 18 岁前的 1000 倍，但 18 岁的人在很多方面仍更强。他期待更数据高效的算法、持续学习、多 agent 交互、新硬件方法，以及促进更文明的人际讨论、帮人找到志同道合者的工具。
- 对职业选择的建议：无论加入前沿实验室还是三人创业，问自己"若最好的结果发生，世界会明显变好，还是只是‘挺酷但无所谓’"——后者不值得投入。找低自我、互补技能、相处愉快的队友；把职业视为不断往工具带里添加新工具。
核心句型 · 10
1. depending on exactly your definition of …, it seems pretty spot-on
“Depending on exactly your definition of junior engineer it seems pretty spot-on I would say”
先用 depending on 限定前提，再下结论，是谨慎肯定的地道表达。适合回应「你的预测准不准」类问题。
2. X is the key to making Y (be) available to …
“Inference is the key to making you know these agent-based systems be available to more and more people”
the key to + 动名词，强调某因素是达成目标的关键；to 后接名词/动名词而非动词原形，是常见易错点。
3. not just A but B
“Can run not just for an hour or two hours on a problem you care about but for some problem domains … for days or weeks”
用 not just … but … 强调程度跃升。仿写时注意两部分结构对称（都是 for + 时间）。
4. if you squint at a problem and think about … from first principles, you can come up with …
“If you just squint at a problem and you think about … how you would solve it from first principles, you can come up with really good ideas”
条件句 + squint at（换角度粗看）+ from first principles，描述创新思维方式，可用于讨论方法论。
5. look for something where X succeeds 0% or 1% of the time, not 20%
“Look for something where the model succeeds 0% or 1% of the time not 20%”
用具体数字对比给出可操作标准，比抽象形容词更有说服力。仿写建议：给判断标准配一个反例数字。
6. the importance of … has actually gone up because …
“The importance of specifying what it is you want has actually gone up because before you'd be handing it off to a very intelligent human”
go up 表「上升」；actually 提示反直觉结论，后接 because 给理由。适合表达「本以为会减弱、实际增强」的论点。
7. That is the crux of …
“That is the crux of you know from my background a research problem”
the crux of 指问题的核心/症结，正式书面与口语皆可。用于总结一段论证后点出要害。
8. if … and the best possible outcome happens, will the world be a lot better, or will the world go 'eh, that's kind of cool but whatever'?
“If I work on this problem and the best possible outcome happens … will the world be a lot better in some way or will the world go eh that's kind of cool but whatever”
用「想象最优结果」的假设句做自我检验，后半句口语化地拟人化世界的反应。适合表达选题标准。
9. I'm not saying we should go do this, but …
“I'm not saying we should go do this, but you know, that's the kind of thing where you do want to … occasionally question assumptions”
先撇清不是建议，再引出真正想强调的原则，是避免被误解的常用让步结构。
10. you never know when you might come across …
“You never know when you might come across a problem where you need these four specialized tools rather than these three”
表达「未雨绸缪」的理由。come across 意为偶遇；rather than 做对比。可用于论证持续学习的价值。
生词精讲 · 128 · 按出现顺序
bold predictions
phr.
0:07
大胆的预测
spot-on
/ˌspɑːt ˈɑːn/
adj.
0:50
完全准确的，一针见血的（口语）
shine
/ʃaɪn/
v.
0:50
（在某领域）表现出色，大放异彩
automation
/ˌɔːtəˈmeɪʃn/
n.
1:44
自动化
measurable objective
phr.
1:44
可衡量的目标
radical
/ˈrædɪkl/
adj.
2:40
激进的，彻底的
shipped in production
phr.
2:40
上线到生产环境（ship 在工程语境中指发布）
remix
/ˌriːˈmɪks/
v.
3:21
重新混合；此处比喻历史以新形式重演
inference
/ˈɪnfərəns/
n.
3:21
推理；ML 中指用训练好的模型做预测
latency
/ˈleɪtnsi/
n.
3:21
延迟，时延
specialization
/ˌspeʃələˈzeɪʃn/
n.
3:21
专用化，专门化
general purpose
adj.
3:21
通用的
inklings
/ˈɪŋklɪŋz/
n.
4:27
模糊的迹象，苗头
internalized
/ɪnˈtɜːrnəlaɪzd/
v.
4:27
内化，真正消化吸收
napkin math
n.
6:00
餐巾纸估算，粗略的数量级计算
fleet
/fliːt/
n.
6:00
（服务器）集群；原义船队、车队
halved
/hævd/
v.
6:39
减半
fiddling with
phr.
6:39
摆弄，反复微调
back of the envelope calculation
phr.
6:39
信封背面式估算，与 napkin math 同义
low precision dense linear algebra
phr.
7:47
低精度稠密线性代数（矩阵乘法等）
energy efficient
adj.
7:47
节能的，能效高的
foundational
/faʊnˈdeɪʃənl/
adj.
7:47
基础性的，奠基性的
over specialize
v.
8:48
过度专用化
interconnect
/ˌɪntərkəˈnekt/
n.
8:48
（芯片间）互连
brought many many chips to bear on
phr.
8:48
bring X to bear on：把 X 投入/施加于某问题
consequential
/ˌkɑːnsəˈkwenʃl/
adj.
8:48
影响重大的
bottlenecks
/ˈbɑːtlneks/
n.
9:39
瓶颈
an order of magnitude
phr.
9:39
一个数量级（约 10 倍）
squint at
/skwɪnt/
phr.
9:39
眯眼看；比喻忽略细节看整体结构
anchored on
phr.
9:39
被……锚定/束缚
first principles
n.
9:39
第一性原理
cache miss
n.
10:25
缓存未命中
disk seek
n.
10:25
磁盘寻道
bandwidth
/ˈbændwɪdθ/
n.
11:09
带宽
accelerator
/əkˈseləreɪtər/
n.
11:09
（AI）加速器芯片
fall off
n.
11:09
衰减，下降
quietly decides
phr.
12:04
悄然决定（暗中起决定作用）
batching
/ˈbætʃɪŋ/
n.
12:51
批处理
amortize
/ˈæmərtaɪz/
v.
12:51
摊销，分摊（成本）
epochs
/ˈepəks/
n.
13:51
（训练）轮次，遍历整个数据集一次
lossy
/ˈlɔːsi/
adj.
15:02
有损的（压缩）
zoom out
phr.
16:07
拉远视角，从宏观看
consolidated
/kənˈsɑːlɪdeɪtɪd/
v.
16:07
整合，合并
retrieve
/rɪˈtriːv/
v.
16:42
检索，取回
stirred together into a soup
phr.
16:42
搅成一锅汤（比喻混杂不清）
decompose
/ˌdiːkəmˈpoʊz/
v.
17:47
分解，拆解
orchestration
/ˌɔːrkɪˈstreɪʃn/
n.
17:47
编排，协调调度
harnesses
/ˈhɑːrnəsɪz/
n.
18:50
（agent）运行框架/脚手架；原义马具
guidelines
/ˈɡaɪdlaɪnz/
n.
18:50
指引，准则
microbenchmark
/ˌmaɪkroʊˈbentʃmɑːrk/
n.
19:46
微基准测试
populate
/ˈpɑːpjuleɪt/
v.
19:46
填充（数据结构）
cache footprint
n.
20:48
缓存占用
iterate on
/ˈɪtəreɪt/
phr.
20:48
对……反复迭代
go off the rail
phr.
22:01
脱轨，失控
shaky
/ˈʃeɪki/
adj.
22:01
不稳的，靠不住的
compound
/kəmˈpaʊnd/
v.
22:01
（误差）累积、复合加剧
openloop system
n.
22:01
开环系统（无反馈校正）
off the distribution
phr.
22:41
偏离（训练数据）分布
degrade
/dɪˈɡreɪd/
v.
22:41
（性能）下降，退化
brightly lit path
phr.
22:41
灯火通明的路径（比喻熟悉、可靠的路线）
discard
/dɪsˈkɑːrd/
v.
23:43
丢弃
plausible
/ˈplɔːzəbl/
adj.
23:43
看似合理的，可能可行的
proprietary
/prəˈpraɪəteri/
adj.
24:30
专有的，私有的
co-design
v.
25:37
协同设计
compounding
/kəmˈpaʊndɪŋ/
v.
25:37
复利式积累
delightful
/dɪˈlaɪtfl/
adj.
25:37
令人愉悦的
caution
/ˈkɔːʃn/
v.
26:49
告诫，提醒
durable
/ˈdʊrəbl/
adj.
26:49
持久的，经得起时间的
at the forefront
phr.
26:49
处于最前沿
selection criteria
n.
27:22
筛选标准
niche
/nɪtʃ/
adj.
29:04
小众的，细分的
building block
n.
29:04
构建模块，基本组件
protein folding
n.
30:04
蛋白质折叠
leverage
/ˈlevərɪdʒ/
v.
31:07
利用，借力
crisp
/krɪsp/
adj.
31:07
清晰利落的
specs
/speks/
n.
31:07
规格说明书（specifications 缩写）
infer
/ɪnˈfɜːr/
v.
31:45
推断
behavioral differences
n.
32:50
行为差异
scarce
/skers/
adj.
33:51
稀缺的
the crux of
/krʌks/
phr.
33:51
……的关键/症结
steering
/ˈstɪrɪŋ/
v.
33:51
掌舵，引导
esoteric
/ˌesəˈterɪk/
adj.
35:02
玄奥的，深奥难懂的
cobbling together
/ˈkɑːblɪŋ/
phr.
35:02
拼凑起来
just barely possible
phr.
35:02
勉强可能做到的
take as a given
phr.
36:10
视为理所当然
fabrication
/ˌfæbrɪˈkeɪʃn/
n.
36:10
（芯片）制造
bits to flip
phr.
37:25
比特翻转（硬件错误）
error margins
n.
37:25
误差余量
racks
/ræks/
n.
37:25
（服务器）机架
design point
n.
38:30
设计点（一组设计约束下的取舍位置）
redundant paths
/rɪˈdʌndənt/
n.
38:30
冗余路径
rhyme with
/raɪm/
phr.
39:28
与……相呼应、相似
neuromorphic computing
/ˌnʊroʊˈmɔːrfɪk/
n.
39:28
神经形态计算（模拟大脑结构的计算范式）
iterations
/ˌɪtəˈreɪʃnz/
n.
40:09
迭代版本
checkpointing
/ˈtʃekpɔɪntɪŋ/
n.
40:09
设置检查点（保存中间状态以便恢复）
intermixed
/ˌɪntərˈmɪkst/
adj.
40:09
混杂在一起的
obscured
/əbˈskjʊrd/
v.
41:16
被掩盖，被淹没
abstraction
/æbˈstrækʃn/
n.
41:16
抽象（层）
lays out chips
phr.
42:03
做芯片布局（layout）
tackle
/ˈtækl/
v.
43:08
攻克，处理
ripe for
/raɪp/
phr.
44:12
时机成熟，适合……
formally verified
phr.
44:12
经形式化验证的
computationally intensive
adj.
44:12
计算密集型的
neural approximation
n.
44:12
神经网络近似
screen
/skriːn/
v.
45:22
筛选
scrape together
phr.
45:22
东拼西凑（资源）
recursively self-improve
phr.
46:04
递归地自我改进
impediment
/ɪmˈpedɪmənt/
n.
46:04
障碍
a nudge from
/nʌdʒ/
phr.
47:15
来自……的轻推/提示
distillation
/ˌdɪstɪˈleɪʃn/
n.
47:15
（知识）蒸馏
program committee
n.
48:30
（学术会议）程序委员会
desperately
/ˈdespərətli/
adv.
48:30
迫切地
distill from
/dɪˈstɪl/
phr.
49:31
从……提炼出（此处双关）
vigorous
/ˈvɪɡərəs/
adj.
50:09
精力充沛的
strive to
/straɪv/
phr.
51:24
努力争取
trade-off
/ˈtreɪdɔːf/
n.
51:24
权衡，取舍
endeavor
/ɪnˈdevər/
n.
52:36
事业，努力
rewarding
/rɪˈwɔːrdɪŋ/
adj.
52:36
有回报的，有成就感的
low ego
adj.
53:10
自我意识弱的，不自负的
complimentary skills
n.
53:10
互补的技能（正确拼写应为 complementary）
tool belt
n.
54:22
工具腰带；比喻技能储备
rattle off
phr.
55:07
一口气说出，脱口列举
exhaustive
/ɪɡˈzɔːstɪv/
adj.
55:07
详尽无遗的
data efficient
adj.
55:07
数据效率高的
on par with
phr.
56:09
与……相当
continual learning
n.
56:09
持续学习
discourse
/ˈdɪskɔːrs/
n.
56:09
公共讨论，话语
civil
/ˈsɪvl/
adj.
56:09
文明有礼的
理解自测 · 11 题 · 是真懂了，还是以为自己懂
1. Dean 在 2025 年 AI Ascent 做了什么预测？一年后他如何评价这一预测？
他预测 AI 已达到初级工程师的水平。一年后（本期开头，第 0–1 段）他认为预测基本准确（spot-on），因为模型在基于 agent 的长时间编程任务上进步很大。他还坦言低估了两点：处理复杂任务的能力增长得比预期快，以及 agent 系统在编程之外的其他领域也开始大放异彩，后者被他视为未来的重要趋势。
2. TPU 的起源与一次「餐巾纸计算」有关，请复述这一故事及关键数字。
2013 年深度学习语音模型把错误率减半，相当于该领域 20 年的进步压缩到几个月。Dean 估算：若每位 Google 用户每天用语音 3 分钟，需要把服务器集群翻倍，成本难以承受。于是他们设计了只做低精度稠密线性代数的专用芯片 TPU。几年后产出的芯片能效比同期 CPU/GPU 高 30–80 倍，延迟低 20–30 倍（第 7–9 段）。
3. Dean 提出的「AI 版延迟数字」包含哪些量？他为什么说现在衡量一切的单位是能量？
他列举了：加速器主存到片上内存再到乘法单元的带宽、单次乘法的能耗、芯片间互连带宽及可连接的芯片数量、从 500 颗扩展到 10000 颗时网络带宽的衰减（第 13 段）。主持人引述他的关键数据：一次计算约 1 皮焦，而从 HBM 搬运数据的能耗约为其 1000 倍（第 14 段）。这一差距决定了算法和产品的可能性边界，因此能量成为核心度量。
4. Dean 和 Sanjay 写的性能优化 skill 具体做了什么？它反映了 skill 的本质是什么？
该 skill 教模型按顺序执行人工做性能优化的流程：跑微基准测量当前性能、修改代码、重新测量、扩大基准集、测缓存占用，然后迭代（第 23–24 段）。Dean 总结说，这本质上是把人会采用的方法以模型能用的形式交给它。配套证据是公开的 30 页《Performance Hints》文档，有人将其摘要喂给模型后，模型对代码性能问题的推理能力提升（第 25 段）。
5. 为什么 Dean 认为批处理（batching）是一个系统问题而非模型问题？请写出推理链。
推理链是：搬运数据的能耗是计算的 1000 倍（第 14 段）→ 若每个样本单独计算，每次都要付这 1000 倍的搬运代价 → 把多个样本打成一批一起算，可把搬运成本摊薄为 1000/batch size（第 15 段）→ 因此 batching 的存在是硬件能耗约束的产物，理想情况下 batch size 1 也可以训练，只是效率不佳（第 16 段）。副作用是批处理对低延迟场景不友好，这又推动了推理硬件专用化。
6. Dean 如何解释长时 agent 在几十步后「脱轨」的原因？他提出了哪两类对策？
原因是分布外：模型在一整套训练分布上学习，一旦任务偏离其熟悉范围，性能会像多数 ML 模型一样突然衰减，偏离越远越差（第 27 段）。对策一是给模型 skills 和提示，把它留在「灯光明亮的路径」上；对策二是多 agent 并行尝试不同方法，再由另一模型评估，保留有希望的、丢弃跑偏的（第 28 段）。他把后者概括为用推理时算力在解空间中做搜索。
7. 「1% 法则」是什么？为什么模型「部分能做」反而是坏信号？
法则是：测试通用模型在目标任务上的成功率，0% 或 1% 是好信号，20% 是坏信号（第 33 段）。理由是：如果模型已能做一部分，说明这项能力已在模型中萌芽，随着更多数据和更大规模，它很可能在 6–12 个月内变好，创业公司的优势会被通用模型收编（第 31、33 段）。完全做不到的任务才可能有持久护城河，如触达通用模型触达不到的私有数据，或用专门数据训练小众模型（第 34 段）。
8. 当代码都由 agent 写时，Dean 认为什么会成为稀缺能力？他给出了哪些培养方法？
稀缺能力是「品味」——判断该让 agent 做什么问题（第 39 段）。他以研究经验论证：选对问题并解决，远胜于漂亮地执行一个无聊问题，而模型在此未必擅长。培养方法有三：一是经验，做过多种问题后能辨认哪些事拼凑已有方法即可行；二是写下未来 12 个月你认为重要的事，一年后回头核对哪些真的重要、哪些被别人做了；三是做疯狂的思想实验，质疑大家视为理所当然的前提（第 40–41 段）。
9. 量子化学的例子如何支持「自动化科学方法」的论点？关键瓶颈在哪里？
Dean 把 AlphaEvolve 等系统统一为科学方法的自动化：提出实验→实现→评估→整合，关键是把循环延迟压到极低（第 47–48 段）。但瓶颈常在评估器速度：DFT 模拟一个分子要一整晚。同事用模拟的输入输出训练神经近似器，快 30 万倍且精度接近，使筛选 1000 万个分子从六个月变成一顿午饭（第 49–50 段）。这说明学习出来的快速验证模型能改变整个实验循环的节奏，而非单次实验。
10. 有人反驳：「通用模型迟早覆盖一切，专门模型没有前途。」Dean 会如何回应？
Dean 会部分同意也部分反驳。他自己承认通用模型的能力范围在持续扩张，创业者必须评估所做之事是否会在 6–12 个月内被覆盖（第 31 段）。但他以 AlphaFold 为例说明专门模型可以在蛋白质折叠这类领域取得通用模型达不到的准确率，并点名材料科学、芯片设计（第 35 段）；同时指出私有数据和低成本训练的小众模型是通用模型触达不到的两类形态（第 34 段）。因此他的回应是：不是专门模型没前途，而是要用 1% 法则筛选真正处于通用模型能力之外的问题。
11. 把「不可靠晶体管」的思想实验迁移到 AI agent 系统设计上，会得到什么启示？这一迁移是否成立？
思想实验的核心是：分布式系统早已用不可靠部件（会坏的磁盘）通过冗余、编码构建可靠整体，为何不把这一思路下沉到晶体管层（第 42–43 段）。迁移到 agent 上：单个 agent 步骤不可靠、会脱轨，正如第 28 段所说，可用多 agent 并行 + 评估器筛选来构建可靠的长流程，这与「多冗余路径传信号」和大脑「多通路」（第 44 段）结构一致。迁移基本成立，因为两者都是「用更高层的冗余与选择弥补底层不可靠」。但 Dean 也提醒此类思想实验常因充分理由而不成立，agent 层的额外算力成本和评估器本身是否可靠，是需要检验的前提。
下载便签
手机：长按图片保存
← 上一期 · REC_024Capitalism vs. Socialism: A Soho Forum Debate
下一期 · REC_026 →MIT Godel Escher Bach Lecture 1
苏菲拉底
THE SOPHIE LAB · ASK THE BEST MINDS THE BIG QUESTIONS
内容仅供学习 · thesophielab.com