1 00:00:00,380 --> 00:00:06,440 如前所述，我是 Lee。今天我们来聊聊这类新型的、始终在线的 So as mentioned, I'm Lee We're going to talk a little bit today about how this new category of always-on 2 00:00:06,580 --> 00:00:12,660 主动式持久智能体是怎么运作的，并带大家了解构建它们的基本模块 proactive persistent agents work and kind of give you some of the understandings of the building blocks for how these 3 00:00:12,680 --> 00:00:18,550 先打个趣：今天讲的一些内容还在开发中 are made just as a, you know, fun statement up here some of the things we'll talk about are in progress 4 00:00:18,610 --> 00:00:25,690 是我们在 SpaceX 正在做的。第一周你们都搭建过一个 we're working on at SpaceX and in week one you all built this very kind of 5 00:00:26,790 --> 00:00:30,210 很简单的智能体框架(harness)，帮你理解核心思想：你有一个模型 Simple harness that helps you understand the core idea which is that you have a model 6 00:00:30,550 --> 00:00:33,550 它能循环调用工具，这其实非常强大 it can call tools in a loop and that's actually very powerful 7 00:00:33,790 --> 00:00:38,870 它为模型解锁了很多新能力，但也有一些缺点 It unlocks a lot of new capabilities for the models but it does come with some drawbacks 8 00:00:38,910 --> 00:00:41,730 最明显的是：你一合上笔记本，进程就停了 Notably you close your laptop and the process stops 9 00:00:41,850 --> 00:00:48,840 它不再运行了。我想讲讲如何把这个强大的简单思想用得更充分 It's no longer running and I want to talk about how you can take this simple idea it's very powerful for how to get more out of models 10 00:00:49,180 --> 00:00:53,820 并扩展成一种新型智能体：始终在线的智能体 and extend it to this new type of agent which is an always-on agent 11 00:00:53,880 --> 00:00:57,020 它不只在你的终端里运行，还能在服务器上运行 So rather than just running in your terminal this can run on a server 12 00:00:57,160 --> 00:01:01,480 能在云端运行。你可以关电脑，它能重启，也能挺过崩溃 It can run in the cloud You can close your computer it can restart. It can survive crashes 13 00:01:01,620 --> 00:01:07,470 比如，这个智能体有自己的电脑，不是你的笔记本，而是一台 Linux 机器 for example This agent has its own computer so not just your laptop but a Linux machine 14 00:01:07,770 --> 00:01:13,210 在云端，有自己的桌面、文件、浏览器等等，也许还有 3D 应用 in the cloud where it has its own desktop files browser, and more, maybe some 3D apps 15 00:01:14,150 --> 00:01:20,830 它能在机器上存文件，随时间更了解你的进展，并积累记忆 It can save files on the machine it can understand more about your progress over time and build up a memory 16 00:01:20,990 --> 00:01:28,410 记住你和它的协作方式。终端里的框架会回应你发的每条消息 of how you work with it Unlike a harness in the terminal that is responding to every message that you send 17 00:01:28,890 --> 00:01:34,190 而这种始终在线的智能体更像同事，不会总是立刻回你消息 this type of always-on agent is more like a colleague where it doesn't always immediately send you a message 18 00:01:34,210 --> 00:01:38,090 它知道什么时候该安静。有时它只想在后台默默干一会儿 It knows when to be quiet Sometimes it might just want to work in the background a little bit and 19 00:01:38,590 --> 00:01:40,750 只有在有重要进展时才回来找你 only come back to you when there's an important update 20 00:01:41,570 --> 00:01:45,950 而且触发它的不只是你打字。不只是你在 And it can be triggered more than just you typing. So it's not just you prompting it in the 21 00:01:45,990 --> 00:01:52,780 终端里给它提示词，也可能是一条 Slack 消息、一封邮件或其他事件唤醒它去干活 terminal Maybe there was a Slack message or an email or some other type of event that wakes up this agent to go do work 22 00:01:54,260 --> 00:02:00,780 理想情况下，我们希望这类始终在线的智能体能长时间持续工作 And ideally The hope with these type of always-on agents is that they can do work over very long periods of time 23 00:02:01,240 --> 00:02:06,960 不只是几小时或几天，甚至几周或几个月，这就需要稍微不同的架构 not just hours or days but maybe even weeks or months and that just requires a little bit of a different architecture 24 00:02:07,280 --> 00:02:13,460 我们接下来会讲。我们会比较深入地讲我们怎么搭建这个架构 which we'll get into So we're going to go decently in-depth on how we built this architecture 25 00:02:13,780 --> 00:02:18,770 怎么搭建智能体框架，是什么让我们走到今天，以及未来的方向 how we built the harness kind of what's got us here today and where we're going in the future 26 00:02:19,210 --> 00:02:24,890 还会讲我们如何把这个系统设计得主动 and we'll get into some of the specifics of how we've engineered this system to be proactive 27 00:02:24,990 --> 00:02:29,550 持久，并且非常听从指令。当然，最后留时间提问 to be persistent, and to follow instructions very well and, of course time at the end for questions 28 00:02:29,930 --> 00:02:34,710 如果大家想问相关的或今天没讲到的问题都可以 if you all want to ask anything about this or anything outside of, what we've covered today 29 00:02:35,780 --> 00:02:43,860 首先，我们是怎么走到这一步的。到目前为止，开发者用 AI 大概经历了四个时代 So first, how we got here There's kind of been four eras of developers working with AI so far 30 00:02:44,000 --> 00:02:51,640 我觉得。第一个是从 ChatGPT 复制代码，粘贴到你喜欢的编辑器里 I would say. The first one was copy-pasting code out of ChatGPT and putting it into your editor of choice 31 00:02:51,660 --> 00:02:55,840 感觉像是几十年前的事了，但其实只是几年 which feels like, you know, decades ago at this point but it was really only a few years 32 00:02:55,880 --> 00:03:03,240 前。后来模型变强了，能用上我们说的那种简化版框架 ago and then once the models started to get better at being able to have kind of that simplified harness we talked about 33 00:03:03,760 --> 00:03:10,520 就是上节课讲的，它们能编辑文件、运行 shell 命令 in, you know, in the last, lecture they were able to edit files run shell commands kind of 34 00:03:10,620 --> 00:03:15,490 有了在终端里与它们协作的简单界面，这一下就火了，因为 Have this very simple interface for how you work with them in the terminal and this really took off because 35 00:03:15,810 --> 00:03:20,190 模型突然有了新能力，能主动帮你干更多活 all of a sudden the model had this new capability where it could go do more proactive work for you 36 00:03:20,630 --> 00:03:28,910 让你更高效。第三个时代，是把最初很简单的编程智能体 and be more you know productive Then this third era was taking the coding agents and where they were kind of simple to start 37 00:03:29,010 --> 00:03:34,070 做成专门的应用，这时大家开始并行运行很多智能体 and giving them these dedicated apps and at this point people were starting to run many agents in parallel 38 00:03:34,410 --> 00:03:38,750 也许在云端而不只在本机运行，但这个阶段 maybe they're running them in the cloud not just on their machine but still at this point 39 00:03:39,170 --> 00:03:43,030 你还是瓶颈，因为你要审查它们运行的 you were kind of the bottleneck in that you were reviewing every command that they were 40 00:03:43,070 --> 00:03:47,600 每条命令，你要确保每条 shell 命令都对，你得一直盯着它们 running you wanted to make sure every shell command was just right you're kind of really watching them and being 41 00:03:47,620 --> 00:03:53,020 每一步都要参与。而现在随着模型、框架和产品的进步 in the loop on everything and now with improvements in models in harnesses and products 42 00:03:53,300 --> 00:03:58,560 我们进入了新时代：开始有始终在线的智能体，能在 we're now in this new era where you're starting to have these always on agents and they can kind of run ambiently in 43 00:03:58,580 --> 00:04:03,380 后台默默运行，有自己的电脑、自己的记忆，能在工作中 the background where they have their own computer they have their own memory they learn new skills on the 44 00:04:03,440 --> 00:04:08,000 学会新技能，在你睡觉或忙别的事时继续工作 job and they keep working after you go you know go to bed or go do other work 45 00:04:08,280 --> 00:04:14,550 一个好例子是：我们已经不再每天都要给智能体发提示词了 A good example of this is we've kind of moved away from this world of prompting an agent every single day 46 00:04:14,630 --> 00:04:19,930 虽然有时还会，但你可以设置智能体在后台自己运行 although we still do that sometimes but you can set up agents where it will kind of just run in the background 47 00:04:20,190 --> 00:04:24,910 以前是：你给智能体发提示词，等着，再发，再等 So maybe in the the before times, it was you would prompt an agent you would wait, prompt an agent, you would wait 48 00:04:25,390 --> 00:04:30,270 它要你运行 shell 命令时，你还得批准或拒绝 and if it asked you to run a shell command you'd have to approve or you know, disallow that command 49 00:04:30,890 --> 00:04:35,150 而现在，你不用排队发命令，可以随时引导智能体 and now, instead of kind of queuing commands you can steer the agent 50 00:04:35,830 --> 00:04:39,250 你可能说“去看看这个帖子”，紧接着又说“哦对了 And you might say hey, go check this thread and then immediately after oh, by the way, you know 51 00:04:39,490 --> 00:04:44,740 其实周四再做吧”。以前的模型会被搞糊涂 actually go do that on Thursday And prior generations of models would get kind of confused 52 00:04:44,810 --> 00:04:49,320 当你立刻打断它、给它别的指令时。但现在它们能理解，就像 when you would immediately interrupt it and give it some other command but now they kind of understand just like 53 00:04:49,340 --> 00:04:53,940 你跟朋友絮絮叨叨说话时，它们能明白这两件事是相关的 how you might ramble on talking to a friend and they can kind of parse that these two things are related 54 00:04:54,440 --> 00:04:56,900 现在的模型和框架都能处理这种情况 The models and the harnesses can now handle that 55 00:04:56,900 --> 00:05:04,200 如果你说“去发这封邮件、去发这条 Slack 消息”，还会有额外的安全检查 And if you say hey, go send this email go send this Slack message there are additional safety checks in place 56 00:05:04,320 --> 00:05:09,140 很多公司称之为“自动审查”，本质上是另一个 AI 模型 which a lot of companies are calling auto-review which is essentially a separate AI model 57 00:05:09,500 --> 00:05:11,700 去检查所有要运行的命令 that's checking all of the commands that get run 58 00:05:12,060 --> 00:05:19,290 有趣的是，这其实比人疲劳地盯着几百条 So, you know, kind of funnily enough that's actually more secure than the fatigue of having to watch hundreds of 59 00:05:19,330 --> 00:05:24,110 shell 命令逐条批准更安全。我们已经到了这样的地步：如果做盲测 shell commands and approve each one We've kind of reached the point where if you did a blind test with what 60 00:05:24,150 --> 00:05:28,270 比较 AI 模型审查命令和人类审查命令，大多数时候 the AI model's checking on the commands and what a human's checking on the commands most of the time the 61 00:05:28,270 --> 00:05:35,690 模型其实做得更好。有点反直觉，但随着模型变强 the model's actually doing a better job Kind of paradoxically, but as the models have gotten better 62 00:05:35,810 --> 00:05:39,380 界面反而变得简单多了 the interface has gotten quite a bit more simple actually 63 00:05:39,380 --> 00:05:46,860 这种始终在线的智能体我们叫它“机器人(bot)”，你会发现这类新 And this type of always-on agent we're calling a bot but as you'll notice there's actually quite a few of these new types 64 00:05:46,880 --> 00:05:49,940 产品最近冒出来不少，而且很受欢迎 of products popping up right now and gaining a lot of traction 65 00:05:50,200 --> 00:05:53,760 我觉得主要原因是：我们都会发短信 And I think the main reason is because we all know how to text 66 00:05:53,760 --> 00:05:57,420 人人都会发消息，大家用手机都很熟练 You know, everyone knows how to text everyone is pretty proficient with using their phones 67 00:05:57,740 --> 00:06:03,130 我们把与 AI 协作的方式提炼成了像给朋友发消息一样 and we've kind of distilled down this way of working with AI that feels like you're just texting a friend 68 00:06:03,530 --> 00:06:05,610 但它真的能帮你干实事 but it can actually go and do real work for you 69 00:06:05,610 --> 00:06:11,790 它不再只是一问一答式地回答问题，而是去完成更长的任务，就像你交给 It's not just answering questions in a Q&A style setting anymore it's actually going and doing longer tasks like you would hand 70 00:06:11,830 --> 00:06:17,810 同事的活一样。具体来说，模型本身有几项变化 off to a coworker for example so there's been a few changes specifically in the models 71 00:06:18,250 --> 00:06:23,390 促成了这一点。我讲六项，但其实还有更多 that have enabled this to happen I'll go over six but there's, you know, quite a bit more 72 00:06:23,830 --> 00:06:29,520 第一，过去一两年里，模型在 The first one is that the models just over the past year or two years have really gotten better at being able to 73 00:06:29,580 --> 00:06:34,780 长时间遵循指令方面进步很大，不会糊涂或出错 follow instructions over a longer period of time where they don't get confused or make mistakes 74 00:06:35,080 --> 00:06:40,800 现在配合合适的压缩和摘要，模型能相当稳定地运行数小时甚至数天 and now models can pretty confidently run for hours or days with the right compression and compaction 75 00:06:40,980 --> 00:06:46,780 这个稍后会讲。第二，模型现在基本能把任务交给助手 which we'll talk about in a little bit Two, models now can essentially hand off to helpers 76 00:06:47,180 --> 00:06:52,610 这些助手叫子智能体。子智能体有用的原因是 and those helpers are called sub-agents And the reason why a sub-agent is helpful is the 77 00:06:52,880 --> 00:06:58,690 模型对当前任务有一份工作记忆，迟早会被填满 the model has kind of its working memory of the process that it's on and that's gonna fill up eventually 78 00:06:59,530 --> 00:07:03,990 所以能交给拥有全新工作记忆的助手就很好 so it's nice if it can hand off to these helpers that have a fresh set of working memory 79 00:07:04,430 --> 00:07:09,530 这样相当于给了它几乎无限的上下文，它能去做那些 and that effectively can give it an unlimited context basically where it can go and do work that 80 00:07:09,570 --> 00:07:14,770 消耗大量 token 或调用很多工具的工作，然后回到主智能体 might be expensive in terms of the tokens it's using or the tools that it's calling and then come back to The main agent 81 00:07:14,990 --> 00:07:21,950 汇报进展，稍后我会放图。第三，模型不仅更擅长调用工具 and report its updates and I'll have some diagrams of that later models now not only are just better at calling tools 82 00:07:22,040 --> 00:07:26,160 几年前它们基本还调不好工具，老是出现幻觉 I mean just a few years ago they basically couldn't call tools very well and they would hallucinate all the time 83 00:07:26,460 --> 00:07:30,280 有意思的是，现在都没什么人谈幻觉了，因为这个问题 It's funny, you don't really hear people talk about hallucinations anymore because it's kind of a solved problem 84 00:07:30,580 --> 00:07:39,120 基本解决了。不仅能调用工具，你还能给它们很强的工具，就像让新人加入团队 mostly But not only can they call tools but you can give them very powerful tools the same way as if you onboarded somebody onto your team 85 00:07:39,460 --> 00:07:45,040 当然包括运行 shell 命令和操作文件，还能给它们一整台 Linux 电脑 so obviously running shell commands and files but also you give them a full computer a full Linux computer 86 00:07:45,740 --> 00:07:52,070 让它们做演示视频之类的。它们能像人一样在屏幕上点来点去 you allow them to make demo videos and such They can click around on a screen just like a human would 87 00:07:52,090 --> 00:07:55,790 而且越来越擅长，这叫“电脑操控(computer use)” and they're getting increasingly better at doing that otherwise known as computer use 88 00:07:56,290 --> 00:08:02,450 意思是任何没有提供 API 或 MCP which basically means that any SaaS product or any website that didn't expose an API or an MCP 89 00:08:02,610 --> 00:08:09,610 服务器的 SaaS 产品或网站，这些模型和机器人都能通过电脑操控自主使用 server can be used headlessly basically so autonomously through computer use by these models and these bots 90 00:08:11,290 --> 00:08:16,250 第四，你可以把所有信息存在文件里，因为对模型来说 You can store all of the information in files and the reason you can do that is because it's actually very easy for the models 91 00:08:16,270 --> 00:08:19,390 回忆和搜索这些信息非常容易，这个后面会讲 then to recall and search this information which we'll talk about 92 00:08:19,460 --> 00:08:25,200 最后，你可以直接示范给机器人看怎么做事，它会把这些变成 And then finally you can just kind of show the bot how to do some work and it will kind of turn that into 93 00:08:25,720 --> 00:08:29,380 技能、记忆或例行任务，之后可以重复使用 skills or memories or routines or things that it can reuse later 94 00:08:30,120 --> 00:08:34,680 关于文件，举个简化的例子，你可能还记得 So on the file thing for example, this is kind of Kind of a simplified example but you might remember 95 00:08:34,820 --> 00:08:40,040 或者可以想象，你用机器人时，它底层可能是这样的 or you might, imagine that when you're working with a bot under the hood it might look something like 96 00:08:40,060 --> 00:08:45,840 你会积累一份用户档案或对用户的理解，它会被放进每一条 this You build up this profile or this understanding of the user and that's gonna be included in every single 97 00:08:45,880 --> 00:08:51,890 提示词里。比如你喜欢全小写、像发短信那样的消息，那每条消息都该这样 prompt. Maybe you like you know, lowercase text-like messaging and every single message should have that 98 00:08:52,470 --> 00:08:57,470 另外，随着你长期使用机器人，你会积累一份工作日志 Also, as you're working with these bots over time you're building up a kind of like a working log 99 00:08:57,610 --> 00:09:02,270 就像同事每天记下自己做过的工作一样 just like if you were working with a colleague and they were writing down notes every day of the work they've done 100 00:09:02,610 --> 00:09:07,390 这些都存进文件，模型之后可以决定去搜索或使用。如果你问 that's all being put into files that the model can then decide to search or use later. If you say 101 00:09:07,430 --> 00:09:10,330 “我上周做了什么？”它就能直接查到 hey, what did I do last week? It just can can go and look that up 102 00:09:10,750 --> 00:09:16,250 模型很擅长用 Linux 命令、运行 shell 命令去 grep 搜索 And the models are very good at using Linux commands running shell commands to go grep and search 103 00:09:16,310 --> 00:09:21,800 翻查这些文件。同样，对于技能，你本质上是 and look through these files similarly, with skills you are essentially 104 00:09:22,600 --> 00:09:27,400 把你做某件事的知识写进一个 markdown 文件 Taking your knowledge of how to do a specific task and encoding it into a markdown file 105 00:09:27,600 --> 00:09:33,560 包括具体流程，以及你认为决定成败的关键 the very specific process you need to do the things that you believe make that you know, successful or unsuccessful 106 00:09:33,940 --> 00:09:38,860 而且不只是你自己的技能，现在有整个生态，很多公司和开发者 And that's not just your skills There's a whole ecosystem now of companies and developers 107 00:09:39,440 --> 00:09:45,160 在写各种技能，比如怎么写好邮件、怎么设计好 API 等等 writing these skills for how to send a great email or you know how to build a great API or many different things 108 00:09:45,780 --> 00:09:49,060 比如在 Grok Bot 里，有一整套插件系统 and, and for example, in Grokbot there is a whole system of plugins 109 00:09:49,080 --> 00:09:53,540 基本上你想用的任何工具都有插件可装，插件用的是技能或服务器 So basically any tool that you want to use there's a plugin you can install which uses skills or servers 110 00:09:54,060 --> 00:10:00,950 还有例行任务，如果你熟悉 cron 定时任务，就是按某个时间表运行提示词 And then also routines which if you're familiar with cron jobs you know you're running some prompt basically on some kind of schedule 111 00:10:01,020 --> 00:10:06,350 这也很有用。总之，一切都是“电脑”，第三部分 which is pretty useful as well So basically everything is computer part three 112 00:10:06,350 --> 00:10:12,450 我们再深入一点，看看这个始终在线智能体的“大脑”，看看幕后 So let's go a little bit further into the kind of the mind of this always on agent or going behind the scenes 113 00:10:14,030 --> 00:10:17,250 智能体“始终在线”到底意味着什么？ What does it mean for the agent to be always on? 114 00:10:17,510 --> 00:10:23,550 这张图从左往右看，一开始服务器处于休眠 You can kind of read this chart from left to right but it starts out with the server being asleep 115 00:10:23,950 --> 00:10:26,970 智能体能用的电脑也处于休眠 and also this computer that the agent can use being asleep 116 00:10:27,090 --> 00:10:33,820 当你发消息，或某个触发事件让智能体启动时 And so whether you send it a message or some kind of trigger decides to make the agent start up 117 00:10:33,860 --> 00:10:39,460 （这个马上讲）就是加载并启动这个智能体的全部状态 which we'll talk about in a second that's basically loading up and booting up all of the state for this agent 118 00:10:39,520 --> 00:10:45,320 它运行在服务器上。如果它需要用电脑，比如用电脑的浏览器 and it's running on a server If it decides that it needs to use a computer maybe it wants to go use the computer's browser or 119 00:10:45,720 --> 00:10:49,460 或者在电脑上点来点去，它也能启动那台电脑 you know, go click around on the computer it can start up the computer as well 120 00:10:50,040 --> 00:10:54,040 用完之后，它还能让它们重新休眠 And then once you're done using it it also has the ability then to basically put those to sleep 121 00:10:54,100 --> 00:11:00,440 为什么这很重要？因为一直开着会很贵，而且世界上也没有足够的电脑 Why is this important? It would be you know, expensive and also there's not enough computers in the world to 122 00:11:00,500 --> 00:11:04,800 给每个人一直开着，服务器的 just have those computers running all of the time for everyone as well as that's a lot of server 123 00:11:04,880 --> 00:11:11,970 用量也很大。所以这些状态可以保存到数据库里：对话记录、记忆、例行任务 usage too And so this state can then be persisted to you know, a database, the, the conversation logs, the memories, the routines 124 00:11:12,050 --> 00:11:15,670 等等。那智能体又是怎么被唤醒的呢？ all of that How does the agent get woken up then? 125 00:11:16,130 --> 00:11:20,210 当然你可以直接给它下指令，但我觉得越来越有意思的是 Well, obviously you can prompt it directly but I think what's increasingly interesting 126 00:11:20,650 --> 00:11:24,950 让其他东西在后台运行，由它们决定什么时候 is having other things running in the background that can then decide when you want to 127 00:11:24,970 --> 00:11:29,820 该去找你的机器人。比如有人在 Slack 上找你，而你的机器人就住在 Slack 里 go reach out to your bots maybe that is somebody pinging you on Slack and your bots can live in Slack 128 00:11:30,220 --> 00:11:33,320 也可能是一通电话，或者你把机器人拉进了一场会议 Maybe that's a phone call or a meeting that you put your bot in 129 00:11:33,460 --> 00:11:39,340 在 SpaceX 这已经越来越常见了：员工自己来不了会议 It's it's increasingly common now at SpaceX where we see bots joining meetings for employees where they couldn't 130 00:11:39,340 --> 00:11:43,240 就让机器人去参会、做笔记，回头再汇报给他们 make it to the meeting but they want their bot to go and just take notes and come back to them 131 00:11:43,540 --> 00:11:48,950 挺有意思的。或者你有些日常任务或后台工作 which is kind of funny Maybe you had some routine or some background work that was taking 132 00:11:49,140 --> 00:11:52,810 要花好几个小时甚至好几天，你希望它做完后给你一份总结 you know, hours or days to do and then you want to come back and give you a summary 133 00:11:52,850 --> 00:11:59,070 这些都会发生，然后我们有一个路由器负责去重，并决定怎么放进对话里 That all happens and then we have basically a router that dedupes that and figures out how to put it into the conversation 134 00:11:59,490 --> 00:12:05,750 它还有一套判断规则：这件事要不要打扰用户？因为不是每件事都 and then it has this heuristic of do I need to bug the user about this or not? Because not everything is actually 135 00:12:05,770 --> 00:12:10,530 真的重要。就像同事在做事时，如果他们每次 important. Just like if you have a coworker working on something it would be pretty annoying if every single 136 00:12:10,530 --> 00:12:13,450 有一点进展就跑来打扰你说“嘿，快看 time they made progress on something they came and bugged you and were like hey, check it out 137 00:12:13,450 --> 00:12:16,460 我这个有进展了”，那会很烦。你其实只需要一个进度更新 like I made progress on this thing You really only need a status update maybe 138 00:12:16,630 --> 00:12:22,680 比如每天一次，看任务而定。我说过智能体有一台电脑 I don't know, every day depending on the task so I talked about how the agent has this computer 139 00:12:23,180 --> 00:12:29,020 一台 Linux 机器，运行在 Firecracker 虚拟机上，这样做非常有用 this Linux box this is running a Firecracker virtual machine and the reason this is really helpful is 140 00:12:29,040 --> 00:12:34,700 因为可以很方便地给机器状态拍快照，关机后再重新启动 because it's very easy then to essentially snapshot the state of the machine and shut it down and start it back up 141 00:12:35,560 --> 00:12:41,300 这台机器上当然有文件，你也可以连进去 And on this machine you of course have files but you can kind of connect into the machine and have 142 00:12:41,320 --> 00:12:47,320 实时查看画面，甚至自己动手操作。机器当然也能运行命令 a live view of the machine and actually control it yourself if you want the machine can of course run commands 143 00:12:47,340 --> 00:12:50,840 如果要跑一些 shell 命令，你可能不想在自己的笔记本上跑 which is great if you're gonna run some shell commands maybe you don't want it on your personal laptop 144 00:12:50,880 --> 00:12:54,220 而是放在另一个安全、受严格限制的环境里 you want it in this other environment that is you know, secured and locked down 145 00:12:54,840 --> 00:12:59,080 另外很有意思的是，它当然装了 Chrome，可以用浏览器 And then another interesting thing is of course it has Chrome and it can use a browser 146 00:12:59,370 --> 00:13:04,470 你还能让它拥有虚拟桌面。所以如果你同时运行五个 but you can also make it essentially have like virtual desktops. So if you're running five bots 147 00:13:04,550 --> 00:13:09,690 十个机器人，每个都要自己的浏览器，就可以给它们各自一块屏幕 ten bots, and each one of them needs their own browser you can essentially let them have different screens 148 00:13:09,990 --> 00:13:14,570 这些都是机器人之间共享的基础设施，它们可以随时调用这些电脑 And all of this again is kind of shared infrastructure between these bots where they can go call on these computers 149 00:13:16,250 --> 00:13:20,530 我们讲了唤醒智能体的各种触发器，就是左边这部分 so we talked about kind of the the triggers that will wake up the agent and that's kind of on the left 150 00:13:20,690 --> 00:13:26,090 比如各种应用、Slack 消息或邮件；中间这块是服务器 so different apps or Slack messages or emails then you have this middle piece which is the server 151 00:13:26,230 --> 00:13:29,950 右边就是刚才讲的那台电脑，可以在上面处理文件 and then on the right is the computer that we just talked about where you can run files and 152 00:13:30,130 --> 00:13:34,350 运行命令等等。下面多讲讲服务器。我之前提过 and commands and other things So let's talk a little bit more about the server. I already mentioned 153 00:13:34,510 --> 00:13:39,090 命令进来后会被放进队列，然后调用紫色方框 you know, commands will come in they get put into this queue and then it calls the purple box 154 00:13:39,230 --> 00:13:46,510 也就是智能体循环。这是上周讲过的：模型在循环里调用工具 which is the agent loop And this is what we covered last week like this is a model calling tools in a loop 155 00:13:46,950 --> 00:13:50,870 关机时，可以把状态保存到数据库或存储里 and then when you shut it down you can kind of save the state to the database or storage 156 00:13:51,890 --> 00:13:55,030 我觉得看这张图挺有意思的，你会发现 so it's like, it's, it's interesting, I think to look at this and realize there's this 157 00:13:55,190 --> 00:13:59,430 核心只是一个小部件，在它上面叠加各种抽象层 one kind of small piece that can then have all of these abstractions built on top to create 158 00:14:00,030 --> 00:14:04,600 就能做出非常强大的产品——只要给它很多好用的工具，并接入 a very powerful product when you give it all of these helpful tools and you connect it to all of 159 00:14:04,620 --> 00:14:12,560 各种服务。智能体跑在服务器上，这在计算领域是很常见的做法 these different services The reason the agent runs on the server is a pretty common practice in computing where you 160 00:14:12,640 --> 00:14:18,780 即“瘦客户端 + 固定服务器”，多年来计算一直在这两种模式间摇摆 want to have kind of a thin client and a fixed server and computing is kind of oscillated between these things 161 00:14:19,080 --> 00:14:23,040 这样做有几个好理由。比如你需要更新 over the years but for example a few good reasons for this is let's say you need to push an 162 00:14:23,080 --> 00:14:28,580 智能体的代码，如果每次都要发应用商店更新才能 update to the you know the agent code it would you know kind of suck if you had to push an app store update just to 163 00:14:28,620 --> 00:14:33,380 把代码推出去就太糟了，而更新服务器可以很快完成 get that code out there versus just pushing a bunch of server updates very quickly also 164 00:14:33,380 --> 00:14:37,870 另外有时需要做繁重的计算，比如语音转写或生成图片 sometimes you need to do heavy computation work you know like transcription or image generation or 165 00:14:37,890 --> 00:14:42,290 这些在服务器上通常比在设备上更快，取决于 something and that's probably going to be faster on a server than on device depending on 166 00:14:42,910 --> 00:14:49,930 是什么设备。而且扩展服务器已经有很成熟的方法，尤其是应对 what the devices are and you know it's there's very established ways that we know how to scale servers especially with 167 00:14:50,290 --> 00:14:58,050 流量和用户增长的波动。但最关键的是：如果把智能体循环 variable traffic and variable growth but the most critical thing here is that if you put the agent loop 168 00:14:58,510 --> 00:15:03,700 放在虚拟机上，你就没法把虚拟机和 on the virtual machine then you can't shut down the virtual machine independently from the 169 00:15:04,670 --> 00:15:10,400 智能体服务器分开关停。所以要把一切存在服务器上，随时能重建 from the agent server so yeah Want to save everything on the server so you can always kind of rebuild 170 00:15:10,720 --> 00:15:15,980 那台虚拟机。比如你需要给虚拟机打一个紧急的 that virtual machine. For example let's say you need to do some critical zero-day Linux 171 00:15:16,640 --> 00:15:21,640 Linux 零日漏洞补丁，那最好能直接重建它，而不是 update on the, on the VM Well it would be great to be able to rebuild that and not you know 172 00:15:21,640 --> 00:15:28,770 把整个智能体循环搞崩、让应用停机。当你按下发送时 blow away the entire agent loop and have downtime in the application when you hit send 173 00:15:28,790 --> 00:15:32,850 这部分可能有点太细了，我简单带过。基本上 this is maybe a a little in the weeds so I'll just kind of gloss through it but basically 174 00:15:33,590 --> 00:15:38,010 如果你想深入学习分布式系统，思考 if you want to dig more into learning more about distributed systems and thinking about 175 00:15:39,110 --> 00:15:43,610 强一致性和最终一致性，这些话题都很有意思 you know, strong versus eventual consistency which are just fun topics to get into you know 176 00:15:43,610 --> 00:15:49,010 你发出一条消息，希望客户端立刻识别并确认收到 you send a message and you kind of want the client to immediately identify that and acknowledge it 177 00:15:49,450 --> 00:15:54,280 然后进入这一轮，模型在循环里调用工具 Then when you go into the turn you're obviously putting these tools in a loop with the model 178 00:15:54,580 --> 00:16:01,180 并存进数据库，而客户端会向所有服务器订阅 saving that to the database and then the clients are essentially putting subscriptions with all of the servers 179 00:16:01,460 --> 00:16:05,320 这样就能知道有没有变化，一直在监听这些信号。收到信号后 so they know if something changed and they're listening for these signals. And if they get a signal 180 00:16:05,380 --> 00:16:09,440 就去服务器拉取最新信息，还可以做一些 then they can go and essentially fetch the latest information from the server and there's some 181 00:16:09,440 --> 00:16:13,560 巧妙的去重，确保如果发生了重试 fancy de-doping you can do to make sure if you know, if there was some kind of retries 182 00:16:14,420 --> 00:16:18,400 列表里不会出现重复的消息 that you would not show you know, multiple messages in the the list 183 00:16:18,400 --> 00:16:21,940 这里要考虑的底层基础设施概念 And so, the, like, underlying infrastructure primitives to think about here 184 00:16:23,640 --> 00:16:31,210 是工作流和任务队列的区别。主要区别是：用任务队列时，假设你一步步往下走 Is a workflow versus a job queue And the main difference here is with a job queue let's say you're kind of going along on steps 185 00:16:31,290 --> 00:16:36,630 你让智能体做一件复杂的任务，它从第一步 you ask your agent to go and do some kind of complicated task and it's making its way from step one 186 00:16:36,650 --> 00:16:42,930 走到第二步、第三步，但突然出了基础设施故障 to step two to step three but then for some reason there was an infrastructure issue there was some problem 187 00:16:43,150 --> 00:16:48,630 普通队列崩了之后重启，你就得重跑第一步 With a normal queue if that crashes and you go to restart it well now you have to replay steps one 188 00:16:48,910 --> 00:16:51,950 第二步、第三步，这显然会带来很多麻烦 two, and three which is obviously that could be pretty problematic 189 00:16:52,290 --> 00:16:57,910 所以你真正需要的是持久化工作流，意思是可以放心重试 So what you really want here is you want a durable workflow which means that it's safe for you to actually retry that 190 00:16:58,270 --> 00:17:01,870 失败后重试时，它会直接从第三步接着做 and when you retry after a failure it picks right up on step three 191 00:17:02,510 --> 00:17:08,550 这在分布式系统和基础设施里某种程度上是已解决的问题 And this is like kind of a solved problem in some ways in distributed systems and in infrastructure 192 00:17:08,570 --> 00:17:13,710 所以我们用了一个开源工具叫 Temporal，感兴趣可以去看看 so we use an open source tool called Temporal which is interesting to look into if you're curious 193 00:17:13,990 --> 00:17:17,100 这样就不用从零重造这些非常复杂的东西 so that we don't have to rebuild a lot of this very complicated stuff from scratch 194 00:17:17,360 --> 00:17:20,900 当然你还是得自己运行它的基础设施，但有些组件可以拼起来用 Of course, you still have to run the infrastructure for it but some of the pieces to put together there 195 00:17:22,740 --> 00:17:28,920 比如来了一条新消息，优先级会不同，取决于 so let's say a new message comes in There's different priorities based on whether 196 00:17:29,520 --> 00:17:37,460 是我在直接给机器人发消息，还是在群聊、Slack 讨论串或后台任务里 I'm messaging my agents or messaging my bots versus if it's in a group chat or a Slack thread or some background routine 197 00:17:37,500 --> 00:17:44,200 举个例子，如果我说“其实把这个活动的日期改一下”，我们希望智能体循环先停下 for example So if I say oh, actually, you know, change the date for this event we actually want that agent loop to stop 198 00:17:44,560 --> 00:17:48,520 我的消息优先于其他正在进行的一切，这很合理 and my message is going to have priority over everything else that's happening which kind of makes sense 199 00:17:49,280 --> 00:17:53,420 其他人的消息就排到队伍后面 for everybody else you're kind of putting things into the queue in the back of the line 200 00:17:53,740 --> 00:17:59,360 如果智能体很快收到三四条消息，它可以把它们合并处理 and if you're, if the, the agent gets, you know three or four messages very quickly it's able to kind of batch those together 201 00:17:59,460 --> 00:18:02,920 变成一轮对话，具体规则你可以自己定 so it's just one turn and you can kind of figure out whatever heuristic you want there 202 00:18:03,820 --> 00:18:06,620 然后它一轮一轮地依次处理 and then it kind of goes through and does each turn at a time 203 00:18:08,220 --> 00:18:13,700 好，下面讲智能体框架（harness）本身，我会拿 Okay, so let's talk about the the harness itself and this is going to kind of take the 204 00:18:14,460 --> 00:18:20,040 上周讲的基础框架，加上一些新工具和新概念 the basic harness that we talked about last week and add some some new tools, some new concepts to it 205 00:18:21,220 --> 00:18:27,510 Grok Bot 的框架和市面上其他类似工具 The biggest difference with the GrokBot harness and probably some of the other tools like this 206 00:18:27,910 --> 00:18:34,410 最大的不同在于我们实现“瘦客户端 + 固定服务器”的方式：只有 on the market is the way we've done the thin client fixed server architecture is there's only 207 00:18:34,690 --> 00:18:39,610 一个工具负责客户端和服务器之间的通信，就叫 send user one tool that communicates between the client and the server and it's just send user 208 00:18:40,350 --> 00:18:48,530 好处是架构非常简单，客户端某种程度上就像在给服务器发短信 And the great thing about this is that it's actually very architecturally simple where the client is just texting the server in some ways 209 00:18:48,810 --> 00:18:52,890 而大量较重的工具都放在服务器上，这样的工具有很多 and you're putting a lot of these heavier tools on the server and there's many of those tools 210 00:18:52,910 --> 00:18:57,630 比如运行 shell 命令、读文件，还有调用云端智能体、录制 There's running shell commands reading files but also calling on a cloud agent or recording your 211 00:18:57,670 --> 00:19:03,960 屏幕或做演示等。我觉得这样效果挺好，因为现在 screen or doing demos for example and this, I, I think, has worked pretty well because now 212 00:19:04,040 --> 00:19:10,240 无论你新开什么客户端——手机应用、服务器、桌面应用、邮件、Slack regardless of where you spin up a new client whether it's a mobile app a server a desktop app, a email Slack 213 00:19:10,500 --> 00:19:13,980 都只靠这一个工具和服务器来回通信 it all just has this one single tool that communicates back and forth with the server 214 00:19:14,380 --> 00:19:18,860 不过在客户端，虽然只有这一个工具，收到工具结果后 On the client, though even though it just has this one tool when it receives the tool 215 00:19:19,080 --> 00:19:24,900 它可以决定用最适合用户的丰富交互方式来展示 it can decide how it wants to display that in a rich and interactive way that makes the most sense for the user 216 00:19:24,980 --> 00:19:29,460 比如服务器返回信息说用户正在登录某个东西 So, if it gets information back from the server that the user's trying to log into something 217 00:19:29,500 --> 00:19:35,860 它就能显示专门的登录表单、连接 1Password 这类工具，也能调用 it can kind of show these unique sign-in forms and connect to tools like 1Password it can go and call on services like 218 00:19:36,460 --> 00:19:43,660 Stripe Link、X Money 等服务，让人付款。你还可以 Stripe Link or use Xmoney or other services to go and allow people to pay for their services. You can 219 00:19:44,180 --> 00:19:49,270 如果要做不可撤销的事，比如发邮件或 Slack 消息 if you're trying to do something that is non is, you can't reverse it sending an email or a Slack message 220 00:19:49,670 --> 00:19:54,430 它可能会回来让你先编辑、批准草稿再发送 it might come back and ask you to you know edit and approve a draft before you send it 221 00:19:54,430 --> 00:19:58,550 有时如果其实不需要回复，它只留个表情回应就行 And then sometimes it just you know you can just leave an emoji reaction if you don't actually need to 222 00:19:58,690 --> 00:20:04,090 回到我刚说的动态工具发现 send a message so going back to what I was saying with dynamic tool discovery 223 00:20:04,770 --> 00:20:11,970 每个模型都有上下文窗口，相当于工作记忆，有 token 数上限 So for, for any model, there's a context window which is kind of a forking memory and that's some limit in tokens 224 00:20:12,550 --> 00:20:20,170 如果把所有工具都塞进上下文窗口，就没剩多少空间了 And if you just jam every single tool possible into that context window you're not gonna have a lot of space 225 00:20:20,780 --> 00:20:25,680 工具的工作方式是：有工具名称、有参数结构 And the way the tools work is you have kind of the the name of the tool you have the schema 226 00:20:25,720 --> 00:20:31,900 还有各种细节。如果你装上 Google Drive 连接器 you have all these different details and if you start installing your Google Drive connector 227 00:20:32,020 --> 00:20:37,820 Gmail 连接器、YouTube 连接器等各种用 MCP 服务器的服务 your Gmail connector, your YouTube connector like all these different services that use MCP servers 228 00:20:38,180 --> 00:20:44,320 它们都有工具，你就会看到像上面那样，很多格子已经被占满 which all have tools you're gonna start looking at something like the top where a lot of the blocks are already filled 229 00:20:45,060 --> 00:20:49,640 问题在于，模型就没有足够空间真正处理对话本身 and the problem with that is it doesn't give the model as much space to actually do the work of the conversation 230 00:20:51,070 --> 00:20:58,050 所以现在很多流行框架都用一个上下文工程技巧 so one kind of context engineering or harness trick that a lot of the the popular harnesses now use is 231 00:20:58,430 --> 00:21:05,350 叫动态上下文，也有人叫它“工具搜索工具”，名字挺有趣 it's called dynamic context Or I think some other folks call it the tool search tool which is a fun name 232 00:21:05,770 --> 00:21:12,090 基本做法是：每次发给模型的消息里只放工具的名字 But basically you're only putting the names of the tools into the every message that gets sent to the model 233 00:21:12,510 --> 00:21:16,800 然后模型再决定：好，我确实要用这个工具 And then the model can decide okay I actually want to go use this tool 234 00:21:16,880 --> 00:21:21,500 就去文件里读取详细信息。说到底一切都是 I can go read the information from a file It's kind of files and a file system all the way 235 00:21:21,540 --> 00:21:28,760 文件和文件系统。这能节省上下文用量，也能省钱 down for all of these things And that just helps you save on your context usage which then also helps save on cost 236 00:21:28,900 --> 00:21:36,200 省用量，对大家都好。总体原则是先试最便宜 save on usage, just better for everybody In general the principle here really is to try the cheapest option or the most 237 00:21:36,270 --> 00:21:42,150 或最可靠的方案，再一级级往上升。最简单的就是 reliable option first and kind of work your way up the ladder here. So the easiest one is just what 238 00:21:42,630 --> 00:21:46,610 智能体根据对话历史已经知道的内容 you know what the agent or what the bot already knows based on the conversation history 239 00:21:47,330 --> 00:21:52,270 如果眼前的上下文里没有，也许有插件或 MCP 服务器 if it doesn't, if it's not in the immediate context maybe there's a plugin an MCP server 240 00:21:52,270 --> 00:21:57,810 或者某个 API，比如从 Plaid 拉取你的银行信息 some kind of API that it can go and pull you know, your, your banking information from Plaid 241 00:21:57,910 --> 00:22:02,210 如果这也不行，可能需要上网搜索，查一下 for example maybe that doesn't work and it needs to go do some kind of web search to see 242 00:22:02,610 --> 00:22:06,090 昨晚小熊队比赛的比分是多少 you know what was the score of the Cubs game last night 243 00:22:07,630 --> 00:22:13,880 如果还不行，就得用电脑打开浏览器去搜索 Maybe if it doesn't have that it actually needs to go to the computer and open up the browser and do some searches 244 00:22:13,920 --> 00:22:18,880 最后，也许需要动用整个桌面，运行脚本 on there And finally like maybe it needs to go and actually use the entire desktop so run scripts or 245 00:22:19,140 --> 00:22:21,680 执行命令，或者在那台 Linux 电脑上构建东西 run commands or build things on that Linux computer 246 00:22:21,680 --> 00:22:27,170 只有这些都失败了，才去打扰用户，请他们提供信息或批准 And only if all of those fail then go bug the user and ask them to go give input or approve something 247 00:22:27,610 --> 00:22:31,370 这样就很像和同事一起工作的感觉：他们不会 And this really helps it feel like what it would be like to work with a colleague. It's like they're not going to 248 00:22:31,410 --> 00:22:35,620 什么事都来烦你，理想情况下会先尝试这些办法 bug you for all these different things Ideally they're going to try a lot of these options first 249 00:22:38,060 --> 00:22:45,300 在那台电脑上，也就是智能体能用的那台 Linux 机器，它大概有三种方式 on that computer, so the the Linux machine that the agent is able to use there's kind of three different ways that it 250 00:22:45,360 --> 00:22:50,700 来获取信息。当然，它可以直接截屏 can retrieve information from it of course it can just take a screenshot of the screen 251 00:22:50,800 --> 00:22:55,480 但还有更高效的办法，它也可以查看网页的 HTML but there's more efficient ways of doing this it could also look at the HTML of the page 252 00:22:55,520 --> 00:23:00,660 还可以查看无障碍树，这能大大减少 but you can also look at the accessibility tree which is a nice way of like cutting down a lot of the 253 00:23:01,220 --> 00:23:06,240 token 用量，帮你找到想让浏览器点击的 token usage and basically to help you find the buttons or links or other elements that you want 254 00:23:07,020 --> 00:23:11,420 按钮、链接或其他元素。当然，如果这些都不行，就直接按像素点击 the browser to click on of course, if none of that works you can just click on pixels 255 00:23:11,800 --> 00:23:16,320 比如有些网站会弹出模态框或对话框，需要你点掉 So some websites, for example have modals or dialogs that pop up that you need to click 256 00:23:16,940 --> 00:23:20,380 所以你的模型、你的智能体也得有这个能力 and so your model your agent needs to have that functionality as well 257 00:23:20,820 --> 00:23:26,740 或者，最坏的情况是回去问用户：这里有个双重验证码 Or, again kind of worst case on that ladder is you go back to the user and like oh there's like a two-factor auth code or 258 00:23:27,160 --> 00:23:35,540 或者验证码之类的。所以把这些拼在一起来看 some kind of captcha or or something like that so the idea is that Kind of putting the pieces together 259 00:23:35,580 --> 00:23:46,190 你有一个主智能体，就像总调度员，主智能体的对话要尽量精简 you have this main agent and it's kind of like your orchestrator and the main agent you want to keep the conversation as sparse as possible 260 00:23:46,710 --> 00:23:52,950 因为它现在可以调用各种助手，而这些助手就是子智能体 because it now has the ability to go call on all of these different helpers and the helpers are just sub-agents 261 00:23:53,350 --> 00:23:58,450 这很有用，因为如果你交给助手去运行一堆命令行指令，或者 And so this is helpful because if you hand it off to a helper to go run a bunch of shell commands or do 262 00:23:58,710 --> 00:24:03,770 做很多昂贵的工具调用，这些工作最终只会变成 a lot of expensive tool calls all of that work that ultimately is ending up in some answer 263 00:24:03,830 --> 00:24:09,400 一个问题或任务的答案，完全不会撑大主智能体的对话和上下文 to a question or a task none of that is bloating the conversation and the context of the main agent 264 00:24:09,420 --> 00:24:16,320 它只带回结果。这就是让产品感觉像拥有无限上下文的方法 It's only coming back with the result And this is how you get to a product that feels like it has infinite context 265 00:24:16,660 --> 00:24:20,560 当然，无限上下文并不存在，每个模型都有上下文上限 Obviously there are infinite context is not a real thing. Every model has a context limit 266 00:24:20,860 --> 00:24:25,280 但有这些技巧和变通办法，尤其是当你训练模型更擅长这些时 but there are these tricks and workarounds such that especially when you train models to get better at this 267 00:24:25,580 --> 00:24:30,180 你就不会感觉撞上了上下文窗口的上限，也不会感觉 that it doesn't feel like you're running up against the limits of the context window and you don't feel a 268 00:24:30,480 --> 00:24:33,980 随着对话推进，模型质量在下降 degradation of the model's quality as you get further along that as well 269 00:24:34,740 --> 00:24:37,530 所以我们有不同的助手，各有不同的技能 So we have these different helpers each with different skills 270 00:24:38,090 --> 00:24:41,010 每一个都是独立的持久化工作流 Each one of these is its own durable workflow 271 00:24:41,010 --> 00:24:45,870 回到队列和工作流的话题，这意味着如果视频助手崩溃了 So going back to the queues versus workflows thing that means if it fails the video helper crashes 272 00:24:46,810 --> 00:24:51,050 比如内存用太多之类的，它可以重启并继续运行 something, you know, uses up too much memory or something that can restart and keep going without 273 00:24:51,390 --> 00:24:57,490 不会丢失进度。而主智能体有点像一个同事 losing progress along the way And then the main agent it it is kind of like a coworker where 274 00:24:57,630 --> 00:25:00,410 它在管理你的待办清单。你可以做几件事 it's managing your to-do list You have a few things you can do 275 00:25:00,410 --> 00:25:02,490 你可以去查看它的进展，可以给它发消息 You can go check on it You can send it a message 276 00:25:02,850 --> 00:25:07,670 比如你还可以停止那些助手。很多设计都参照了 You know, you can stop the helpers for example and so a lot of this is modeled after the 277 00:25:07,810 --> 00:25:12,060 一个工程师团队攻克复杂问题时最理想的扩展方式 ideal way you would scale you know a team of engineers working on a very complicated problem 278 00:25:13,060 --> 00:25:17,840 下面分享几个我们团队在开发中发现的有趣小细节 So, like just a few little interesting tidbits from building this out on the team 279 00:25:19,060 --> 00:25:25,700 后面会多讲讲提示词缓存，但理想情况下，不要在对话轮次中 I think we'll talk a little bit more about prompt caching here but ideally you don't add or remove new tools 280 00:25:25,740 --> 00:25:34,520 增删工具。就像那张满是方框的幻灯片，如果工具描述或结构定义特别长 during turns and just like I mentioned with the slide with all of the boxes, if you have extremely long tool descriptions or schemas 281 00:25:34,560 --> 00:25:39,610 会占用上下文窗口大量空间，对价格和用量都不好 that's gonna take up a lot of space in the context window which is bad for pricing it's bad for usage 282 00:25:39,920 --> 00:25:43,910 理想情况下要尽量精简，这里有很多可以做的优化 Ideally you wanna minimize that so there's plenty of optimizations you can make here and 283 00:25:44,130 --> 00:25:50,650 在工程上把它压缩下来。另一个有趣的点是，你可能听人说 on the engineering side to just trim that down another interesting one is like, you might hear people talk about how 284 00:25:50,830 --> 00:25:54,670 我们可以做一个只有一个工具的智能体框架，这个工具就是运行命令行 oh, like we can have a harness that just has one tool and that one tool is like, running shell commands 285 00:25:55,090 --> 00:25:59,770 这没错，很多事都能这么做，但如果你发现模型 And this is true and you can model a lot of things this way but if you see the model doing 286 00:26:00,070 --> 00:26:04,890 90% 的时间都在为某件事跑命令行，那其实更高效的做法是 you know 90% of its time doing shell commands for this one thing it actually can be more efficient to make 287 00:26:04,930 --> 00:26:10,170 为它做一个专用工具，这样审查产品运行方式的人 a dedicated tool for some of that stuff and it's it's a bit more legible for people actually reviewing 288 00:26:10,520 --> 00:26:16,540 也更容易看懂。最后一点也挺好笑：给人写报错信息时 how the product is working The last one's kind of funny too obviously if you're writing errors for humans 289 00:26:16,660 --> 00:26:20,460 写得详细很有帮助，比如“这一步失败了，请点这个链接” it's helpful to make it very descriptive Like, okay, this thing failed, like click on this link 290 00:26:20,500 --> 00:26:23,840 或者“下一步该这样做”。结果发现这对智能体也很有用 or here's what you need to do next Turns out that's also very helpful for agents 291 00:26:24,000 --> 00:26:28,360 所以为智能体设计好的报错信息，也是一种开发体验 So there, there is some DX in designing good error messages for agents too 292 00:26:30,040 --> 00:26:36,660 当你让机器人长时间替你干活时，非常重要的一点是 And when you're trusting your bots to go and do work over a long period of time it is very important that you 293 00:26:37,060 --> 00:26:42,200 你得信任它能保护好你的信息，运行方式也安全 trust them to be secure with your information to be secure with how it operates 294 00:26:42,320 --> 00:26:48,460 所以即使它在独立的机器上运行、不影响你的电脑，你仍然希望 So even working on its own separate machine not polluting anything on your machine still you would like it if 295 00:26:48,960 --> 00:26:52,260 每条命令行指令执行前，都有一个模型先审查一下 you know, a a model reviews every shell command before it goes 296 00:26:52,660 --> 00:26:57,900 也就是自动审查。还希望那些不可信的输入，比如收件箱里的邮件 So auto review It would be great if you know, when these untrusted inputs like a email in your inbox 297 00:26:58,330 --> 00:27:02,970 可能是钓鱼邮件，或者某个 webhook，你不想把它当成用户 could be a phishing email or some web book you don't want to send that to the model and act like it was a user 298 00:27:03,010 --> 00:27:07,470 指令发给模型。业界在这方面还有很多工作要做，确保 instruction. And there's plenty of work to do here in the industry on just making sure that that is 299 00:27:08,030 --> 00:27:16,690 能正确防范提示词注入。对于不可撤回的操作，比如发邮件 properly protected from, prompt injections For these kind of non-reversible things like sending emails 300 00:27:16,770 --> 00:27:22,090 一定要先问用户。你还可以在产品里设计一些流程 right? You want to really ask the user first And then also you can build in some workflows into the product to keep 301 00:27:22,290 --> 00:27:25,250 让密码等信息不进入模型的对话 passwords and other things outside of the model conversations 302 00:27:26,150 --> 00:27:31,860 好，上下文工程也叫智能体框架工程 Okay Context engineering is also known as harness engineering 303 00:27:31,900 --> 00:27:34,180 我觉得两者差不多，只是随着时间演变了 Like I feel like it's kind of the same thing. They've evolved over time 304 00:27:34,200 --> 00:27:41,260 基本上就是有哪些技巧，能减少这些框架里用掉的上下文 But basically what are some tips and tricks we can do to minimize the amount of context that's being used in these harnesses 305 00:27:42,000 --> 00:27:46,200 这能提高效率、降低成本，还有其他好处 which is good for efficiency for reducing costs, and other reasons 306 00:27:47,440 --> 00:27:51,790 我提过提示词缓存，这里就能看出它为什么这么重要 I talked about prod caching this is a good example of why it's really important 307 00:27:51,830 --> 00:27:58,590 想想你在用编程智能体或任何智能体时，大部分 token So if you think about when you're working with a a coding agent or any agent most of the tokens used are actually 308 00:27:59,030 --> 00:28:04,210 理想情况下都是缓存过的，因为你发出一段对话，然后做个改动 ideally cached tokens because you're sending it a conversation and then you're doing a change 309 00:28:04,230 --> 00:28:08,610 你只是多加一条消息，然后基本上把整段对话重新发一遍 you're, you're adding one more message and then you're basically resending the entire conversation over again 310 00:28:08,950 --> 00:28:13,990 看看模型的定价，缓存过的 token 当然有很大折扣 So if you look at the pricing for models there's huge discounts, of course, on if a token is cached 311 00:28:14,030 --> 00:28:17,670 所以你希望它们尽可能久地留在缓存里 and you want to keep them in the cached window as long as possible 312 00:28:17,670 --> 00:28:24,200 做法就是让工具定义、系统提示词这些信息 And so a way to do this is you try to keep the Tool definitions the system prompts all this information 313 00:28:24,700 --> 00:28:30,800 在不同调用之间尽量保持不变，这样就不会让缓存失效，也不用 as static as possible between different calls so that you don't break the cache and then have to pay 314 00:28:31,080 --> 00:28:36,380 每次都为未缓存的 token 付费。如果顶部是固定的部分 for the uncached tokens every single time So if you have that fixed part at the top then, you know 315 00:28:36,380 --> 00:28:41,350 到了底部只有最新的消息，那就是你唯一需要 when you get down to the bottom and you only have the newest messages that comes in that's the only delta that you're paying the 316 00:28:41,450 --> 00:28:46,470 按未缓存价格付费的增量。这里面也大有学问 uncached tokens for. And there's a a whole art and science to this as well 317 00:28:47,390 --> 00:28:52,650 因为理想情况下，主智能体就是那个和你进行 Basically because that main agent in the most ideal sense this is the one you're having this very 318 00:28:52,730 --> 00:28:58,360 超长对话的家伙，它会压缩、也就是总结上下文 very, very long conversation with that's, compacting or compressing, so summarizing the context 319 00:28:58,670 --> 00:29:03,760 一遍又一遍，有时你甚至察觉不到，后面会举个例子 many, many times over, without you even noticing sometimes which we'll show an example of 320 00:29:03,800 --> 00:29:06,380 所以你真的要让上下文保持精简 So you really want to keep that context small 321 00:29:08,180 --> 00:29:13,560 一个办法是：每当有很长的输出时，不要把它塞进上下文 Some ways of doing that is anytime there's a very long output rather than stuffing that in the context 322 00:29:13,880 --> 00:29:19,370 而是放进文件里。智能体很擅长读文件，所以效果不错 you can just put it in a file And the agents are very good at reading files so this actually works pretty well 323 00:29:19,400 --> 00:29:22,670 就像读取一个技能或读取一段记忆一样 which is just like reading a skill or reading a memory for example 324 00:29:22,890 --> 00:29:30,190 如果要给模型看截图或做其他很昂贵的工作，就交给助手去做 If you give the model a screenshot or some other very expensive work you just delegate that out to a helper 325 00:29:30,510 --> 00:29:38,990 也就是交给子智能体，不影响主智能体的上下文。理想情况下 Again, so it goes to the sub-agent and it doesn't affect the main agent's context and then also, in an ideal world 326 00:29:39,370 --> 00:29:45,480 你还要像打理花园一样，不断修剪、删掉对话里不必要的内容 you want to kind of be always gardening and trimming and removing unnecessary stuff from this conversation 327 00:29:45,560 --> 00:29:51,840 边进行边清理上下文窗口，既为了省钱，也为了记忆效果和模型表现 from this contact window as you're going, both for cost but for also recall and just making sure the model's performing 328 00:29:51,880 --> 00:30:00,580 所以缓存未命中其实就像系统里的一个 bug。如果回到 well So if there's a cash cash miss, it's kind of like a bug in the system really and if, if you go all the way back to 329 00:30:02,200 --> 00:30:07,750 今年一月，那时 OpenClaw 正火起来，人们 All the way back to January of this year when OpenClaw was really getting popular and people were using it 330 00:30:08,310 --> 00:30:15,570 在模型厂商的订阅套餐里用它，这是厂商从没见过的新用法 inside of subscription plans for model providers this was a very new usage pattern that model providers hadn't really seen 331 00:30:15,910 --> 00:30:22,710 智能体框架也还没针对这种工作流优化，所以出现了大量缓存未命中 and the the harness hadn't really yet been optimized for this type of workflow so there was a lot of cash misses 332 00:30:23,070 --> 00:30:28,920 而智能体一直在运行时，这显然是个问题，所以做了大量修复 And when you're doing something like this where agents are running all the time that's obviously problematic, so lots of fixes 333 00:30:29,320 --> 00:30:35,140 来确保尽可能保持提示词处于缓存状态 had to go in to make that robust to keeping the cash keeping the prompt cached as much as possible 334 00:30:36,500 --> 00:30:44,640 一个好玩的技巧是让提示词里的内容保持固定顺序，即使你部署 so like a fun a fun trick here is just trying to keep everything in the prompt in order such that even if you deploy 335 00:30:44,680 --> 00:30:49,180 新改动，或想测试新的系统提示词、加一个新工具 a new change or you want to test out some new change to the system prompt or add a new tool 336 00:30:49,520 --> 00:30:53,570 你几乎可以给每一段算个哈希值，始终保持固定顺序 you can you can almost hash each section and always keep a fixed order 337 00:30:53,980 --> 00:30:57,650 比如你在做 A/B 测试，想在提示词里加一段新内容 Like let's say you're doing a A-B test and you want to put a new piece of the prompt in there 338 00:30:57,650 --> 00:31:02,490 你可不想因此打乱所有其他人的提示词顺序，否则缓存就失效了 you really don't want that to screw up the prompt order for everybody else otherwise you just busted the cache 339 00:31:02,690 --> 00:31:09,090 那就麻烦了。另一个好玩的技巧是，当你打开聊天开始打字时 and then that's, that's problematic another fun one is when you open the chat and you start typing 340 00:31:09,950 --> 00:31:15,810 服务器其实可以提前准备好提示词，先把缓存预热，就像 the server can actually go and prepare the prompt early to basically warm the cache almost like 341 00:31:15,810 --> 00:31:19,490 你在网站上把鼠标悬停在链接上时，它会预先加载下一页 if you're on a website and you go to hover on a link it can go and prefetch the next page 342 00:31:20,170 --> 00:31:28,430 所以有大量这样的小工程优化，层层叠加，决定了模型在 token 用量上能走多远 so there's like tons and tons of little engineering optimizations that compound how far these models can go in terms of token usage 343 00:31:29,080 --> 00:31:32,060 以及长时间使用时它能有多聪明 and how smart they can be when you use them for very long periods of time 344 00:31:32,860 --> 00:31:38,860 关键在于，长时间工作时你必须非常擅长 And the key thing is that when you're working over a very long period of time you essentially have to get very good at 345 00:31:39,080 --> 00:31:44,940 压缩，也就是总结。比如模型有 20 万 token 的上下文窗口 compaction or summarization Because let's say the model has a 200,000 token context window 346 00:31:45,280 --> 00:31:48,840 你一直聊一直聊，最终会到一个需要总结的点 you're going and going and going eventually it's going to hit a point where it needs to summarize 347 00:31:49,140 --> 00:31:57,220 而所有压缩都会丢信息，所以要找到最好的算法，不丢掉重要细节 and all compression is lossy So you have to figure out the best algorithm to do the compression and not lose important details 348 00:31:57,580 --> 00:32:02,750 我敢说光这个问题就能做好几篇博士论文，真的非常难 And there's, I'm sure, entire PhDs just for this problem like it's very, very tough to do 349 00:32:02,830 --> 00:32:07,390 对吧？举个例子，假设你和机器人来回聊天 right? But just to show an example like let's say you're going back and forth with your bot 350 00:32:07,950 --> 00:32:11,710 已经超过 10 万 token，然后你去干别的了 you have over 100,000 tokens and then you go and do something else 351 00:32:12,210 --> 00:32:17,410 模型厂商有一个缓存时间窗口，在这段时间里缓存 Well there is this time period where the model providers have like a cache window when the cache is 352 00:32:17,450 --> 00:32:20,110 还是热的，可能是 10 分钟，也可能是 60 分钟 still warm. Maybe it's 10 minutes maybe it's 60 minutes 353 00:32:20,770 --> 00:32:25,790 我们用的一个很有用的技巧是：趁缓存还热的时候 One trick that we do which I think is really helpful is while you're still within the warm cache period 354 00:32:26,350 --> 00:32:30,650 马上做压缩，因为你有这么长的对话 you should probably do the compaction right now because you have this long conversation 355 00:32:30,690 --> 00:32:35,740 现在做会便宜得多。所以先压缩好，等用户 it's going to be much cheaper So Go ahead, do the compaction and now when the user comes back 356 00:32:36,120 --> 00:32:39,460 30 分钟后回来时，就从这份摘要开始 in 30 minutes or whatever they're starting from this summary 357 00:32:39,940 --> 00:32:44,350 在摘要基础上继续加消息，对他们来说便宜多了 And the summary now is going to be much cheaper for them to continue adding messages on 358 00:32:44,410 --> 00:32:50,790 要把总结算法做得高质量，还有很多工作要做 So lots of work to do there to make it to make the algorithm very high quality for how you do the summarization 359 00:32:51,550 --> 00:32:56,610 说到压缩，以及选择把哪些信息放进智能体的记忆 When you talk about compaction and what information you choose to include in the agent's memory 360 00:32:57,010 --> 00:32:59,470 我把它分成几个层次来考虑 I kind of think about it in a few different layers 361 00:32:59,950 --> 00:33:05,760 第一层是关于用户的重要事实，每次都必须包含 The first one is like facts about the user that are very important to be included every 362 00:33:05,800 --> 00:33:13,420 回到前面说的，用户总想要全小写文字，这大概每条消息都得带上 single time Going back to earlier when I said oh like the user always wants to have lowercase text. That probably needs to be in every single message 363 00:33:13,480 --> 00:33:18,680 否则如果你走全小写风格，突然某条消息出现大写字母就很怪了 otherwise it would be really weird if one message now has uppercase letters if you're going like full Sam Altman style 364 00:33:19,960 --> 00:33:24,380 然后你可能还有一些想放进去的提示或事实 Then you probably also have some prompts that you want or some facts that you want to include 365 00:33:24,460 --> 00:33:31,290 有很多方法设计算法，来判断哪些事实最值得保留 in there And there's many different ways you can make this algorithm to determine what are the most important facts to keep 366 00:33:31,770 --> 00:33:36,490 理想情况下，要对这些事实做一种“垃圾回收”，仔细想想 So ideally there's this kind of garbage collection of this fact and like if you think about it 367 00:33:36,730 --> 00:33:40,350 我们的大脑就是这样运作的：这个事实现在值得记住 this is kind of how our brains work It's like this fact right now is important to keep 368 00:33:40,390 --> 00:33:45,990 它和今天的对话有关。你现在大概不会想着一场足球赛 it's relevant for this conversation today You're probably not thinking about you know, a soccer match right now 369 00:33:46,590 --> 00:33:51,450 大脑很擅长把当下相关的事放在脑子里，留到以后再回忆 And your brain is very good at keeping those ones in context right now and kind of remembering them for the Later 370 00:33:51,890 --> 00:33:58,110 我们在这里也尝试模拟了一些，智能体框架当然有这些核心事实 we tried to model some of that here where the the, the harness has, of course these core facts 371 00:33:58,320 --> 00:34:04,220 它还有按日期记录的日志，基本上就是你过去的对话历史 It also has these dated logs of kind of your past conversation history essentially 372 00:34:04,220 --> 00:34:08,420 然后还有这些草稿笔记，就是你一直在记的一些小事 And then also these kind of scratchpad notes So it's like little things that you've been 373 00:34:08,420 --> 00:34:13,580 手头正在做的事，这些是临时的，之后会消失，但可能只在 working on right now that are designed to be ephemeral and they will go away but they might be relevant just for the 374 00:34:13,620 --> 00:34:18,180 接下来半小时或一小时里有用。其他所有东西都能通过文件找到 next 30 minutes or an hour And everything else can be found through files 375 00:34:18,500 --> 00:34:24,450 这就是文件的魔力：模型很会执行命令，随时都能自己去查资料 This is the magic of files of the models being good at running commands is like they can always just go look stuff up 376 00:34:24,950 --> 00:34:28,730 而且它们查得非常好。那么，这一切会走向哪里？ and they're actually very good at doing that So where, where is this going? 377 00:34:29,410 --> 00:34:36,030 有一个“飞轮”：训练模型让它更擅长这个产品。简单 There's this flywheel of training the model to get better at the product and just as a 378 00:34:36,490 --> 00:34:43,650 复习一下训练大语言模型的几个阶段：首先是预训练，模型从 high-level refresher on kind of the stages of training a large language model first you have pre-training where the model learns 379 00:34:44,170 --> 00:34:49,260 海量网络文本或其他私有数据中，学到对世界的总体理解 a general understanding of the world from large amounts of internet text or other private 380 00:34:49,260 --> 00:34:54,460 然后是监督微调，教模型按某种方式表现 data Then you have supervised fine-tuning where you're teaching the model how to behave 381 00:34:54,980 --> 00:34:58,940 比如当聊天助手，或者当智能体 in a certain way maybe as a chat assistant or as an agent 382 00:34:59,440 --> 00:35:02,980 最后是强化学习，有点像教模型 And then finally you have reinforcement learning where it's kind of like teaching the model 383 00:35:03,000 --> 00:35:09,060 玩游戏，并越玩越好。对于这种“常驻”智能体来说 how to play games and get better at those games and in the context of these always-on agents 384 00:35:09,420 --> 00:35:14,920 很有帮助的做法是：在监督微调中让模型看智能体框架(harness)的示例， it's very helpful, for example to train the model to see examples of the harness and to know 385 00:35:14,980 --> 00:35:20,450 学会在框架里工作，从而理解你给它的有哪些工具 how to work inside of the harness in SFT so it can understand the tools that you have available 386 00:35:20,860 --> 00:35:25,770 在强化学习中，训练模型更好地遵循指令也很有用 It's helpful to train the model to be better at instruction following during reinforcement learning 387 00:35:26,050 --> 00:35:32,650 比如有技能(Skills)时，懂得该调用哪个； how to understand the right skill to call when there's Skills for example how to click on the right spots when 388 00:35:32,930 --> 00:35:38,950 在浏览器里需要到处点击时，知道点哪里。不只是训练本身 you're given in the browser and needs to kind of click around and not only the training itself 389 00:35:39,070 --> 00:35:41,750 你怎么部署模型、怎么做推理，也很重要 but also how you serve the model the inference that you do 390 00:35:42,290 --> 00:35:47,250 你对话的主智能体，最好延迟很低，能马上回应你 The main agent you're talking with you know ideally you want it to have very low latency. You want it to come back to you right away 391 00:35:47,590 --> 00:35:51,530 回复得特别快，你可以调整框架和推理来做到这一点 give you really fast replies and you can tune the harness and the inference to do that 392 00:35:51,890 --> 00:35:56,430 但当你把任务交给一个助手，让它去干一段 But then when you go hand it off to a helper and you're asking it to go do some work for a very 393 00:35:56,490 --> 00:36:01,190 很长很长的活时，十分钟和十二分钟就差别不大了 very long period of time the difference between ten minutes and twelve minutes is kind of negligible at that point 394 00:36:01,290 --> 00:36:10,350 这样你就能控制如何优化。这个飞轮的运作方式是： so that allows you to kind of control how you optimize things the way that this kind of flywheel works is that 395 00:36:10,490 --> 00:36:16,240 假如你的智能体做错了事，最好把它变成某种评测 let's say your agent gets something wrong ideally then that gets turned into some kind of evaluation 396 00:36:16,450 --> 00:36:23,640 比如模型不小心调用了不该调用的工具，那在后端 So, the model, you know, accidentally called this tool and it wasn't supposed to then on the back end 397 00:36:23,660 --> 00:36:28,760 你可以去修框架、修 bug、改提示词，也可能是推理的问题 you can go and fix the harness you can fix the bug you can fix the prompt maybe it's an inference issue 398 00:36:29,560 --> 00:36:32,700 你在评测里确认它真的修好了，然后发布更改 you can check and make sure it's actually fixed in the evals and then ship the change 399 00:36:33,320 --> 00:36:39,200 做这些产品的团队，几乎每天都在这样做 and, you know, the, the teams, and the products, the teams working on these products they're kind of doing this every single day 400 00:36:39,240 --> 00:36:42,040 找 bug、修 bug、用评测来衡量 They're finding bugs, they're fixing it they're measuring it with evals 401 00:36:42,280 --> 00:36:44,500 这就是一个不断变好的“内循环” That's kind of this inner loop that you're always getting better at 402 00:36:44,740 --> 00:36:48,620 然后还有“外循环”：每隔几个月或每个月， But then there's this outer loop which is like every few months or every month or however long 403 00:36:48,950 --> 00:36:54,230 你会训练新模型。理想情况下，新模型会从这些 bug 和失败中学习 you're training new models And ideally the new models are learning from all of these bugs and these failures 404 00:36:54,650 --> 00:36:58,530 并在评测上不断提升，在你关心的事情上变得更好 and they're kind of climbing on the evals to get better at the things that you care about 405 00:36:58,530 --> 00:37:01,750 你可以把新版本放进产品里，确认它们 And you can drop the new versions into the product and make sure that they're improving 406 00:37:01,790 --> 00:37:05,630 在各方面都有进步。有趣的是，新模型出来时 at all these different things And the fun thing about this is then when the new models come out 407 00:37:05,650 --> 00:37:10,010 会有能力上的飞跃，让你能用新方式使用产品 they have new capability jumps it allows you to use the product in new ways 408 00:37:10,410 --> 00:37:15,970 比如，关于在群聊里使用机器人的训练数据其实并不多 So, there hasn't really been a lot of training data for example on using bots in a group chat 409 00:37:16,090 --> 00:37:20,160 这是一种刚刚出现、正在摸索的事，做对很重要 This is kind of an emergent thing that's being figured out and it's important to get it right 410 00:37:20,240 --> 00:37:26,000 显然，像 Moltbook 或 Hugging Face 事件那样，事情也可能搞砸 Obviously if you Seeing things like mold work or the hugging face incident like there are ways that that goes poorly 411 00:37:26,480 --> 00:37:31,300 所以要能训练模型在多智能体系统里表现得好， so it's important to be able to train the models how to behave well in a multi-agent system where 412 00:37:31,800 --> 00:37:38,460 比如不同的机器人互相对话时。理想情况下，你要一直跑这些 different bots are talking to each other for example and so ideally you're always kind of running these inner 413 00:37:38,500 --> 00:37:42,620 内循环和外循环——作为一个在公司里打造这类框架的产品团队 loops and outer loops as a product team building these type of harnesses at a company 414 00:37:43,020 --> 00:37:49,800 一起做 AI。我们在 Cursor、现在在 working on, on AI together one of the principles that we had at Cursor and now 415 00:37:49,800 --> 00:37:56,810 SpaceX 的一个原则是“删掉产品”。意思是你得内化并假设 at SpaceX is to delete the product And what we mean by this is you kind of have to internalize and assume 416 00:37:57,170 --> 00:38:03,790 今天的模型是它们最差的时候，所以六个月后你可能要彻底重做界面 that the models today are the worst they'll ever be which means that in six months you might need to completely rebuild the UI 417 00:38:04,110 --> 00:38:08,890 我今天讲的大部分内容，跟我 Like most of the stuff that I've talked about today it's a pretty big update from if I would 418 00:38:08,890 --> 00:38:13,310 六个月前、更别说一年前讲的相比，变化都很大，因为我们学到了很多 have given this talk six months ago or definitely a year ago because we've learned a lot of 419 00:38:14,250 --> 00:38:18,130 构建框架和做上下文工程的新实用方法 new practical ways of building harnesses and doing context engineering 420 00:38:18,770 --> 00:38:24,290 所以好的策略是：为下一代模型而构建 So a good strategy here is you really want to try to build for the next generation of the model 421 00:38:24,390 --> 00:38:31,630 尽量少做东西，只做一个非常非常简单的界面来使用这些模型 Like how do you build as little as possible and get by while making just a really really simple interface to work with these models 422 00:38:31,670 --> 00:38:35,190 让它们替你思考和行动 and allow them to to, to think and act for you? 423 00:38:36,070 --> 00:38:43,720 我认为未来六到十二个月会发生一些事，这些算比较稳的押注 So some things that I think will happen in the next six to twelve months we'll see. I think these are Decently safe bets 424 00:38:43,800 --> 00:38:48,700 现在，你的智能体或机器人可能工作几个小时， but, you know, right now let's say you have your agents or your bots working for maybe a few hours 425 00:38:49,100 --> 00:38:56,570 或一天，看任务难度。我觉得很快它们可能会工作好几周 maybe a day depending on how tough it is I think it's very possible that pretty soon your agents will be working for weeks 426 00:38:56,990 --> 00:39:02,470 甚至几个月，这就带来一些有趣的问题：我们该如何训练和评测 or months and it does bring up some kind of interesting questions in terms of how we train and evaluate 427 00:39:02,490 --> 00:39:09,710 这些模型。我认为模型会越来越擅长回忆 those models I think the models are gonna get better and better about how they recall on past information in 428 00:39:10,070 --> 00:39:15,680 过去对话里的信息。研究中有很多不同方法，比如用什么算法， in conversations Plenty of different approaches in research here for the type of algorithms used or the 429 00:39:16,120 --> 00:39:20,400 底层数据怎么表示——也许是图，也许是文件，也许是别的 representation of the data under the hood Maybe it's a graph maybe it's files, maybe something else 430 00:39:22,040 --> 00:39:25,980 理想情况下，如果你有这样一个“数字同事”，就把你所有的工具都给它 I think ideally if you have this digital colleague you give it literally all the tools that you have 431 00:39:26,660 --> 00:39:31,600 就像给一位真人新员工做入职一样。当然，很多组织还在慢慢接受这个想法 if you were onboarding a real human And so I think organizations are still warming up to that idea 432 00:39:31,720 --> 00:39:34,380 所以这方面还有些事要做 of course So there's a little more to do there 433 00:39:34,900 --> 00:39:37,570 我觉得界面可能会变得更简单 I think the interfaces will maybe get even easier 434 00:39:37,720 --> 00:39:44,950 有些人会更喜欢直接用语音和他们的机器人或智能体对话 I think some people will prefer to just talk to their kind of bots or agents as a, in a speech-to-speech way 435 00:39:45,050 --> 00:39:50,430 也许他们会有个 Jony Ive 设计的设备，希望它很酷，能跟它聊 Maybe they'll have their like, Johnny Ive device that is hopefully really cool and they can, like, talk to that 436 00:39:51,250 --> 00:39:59,390 一整天。我认为模型和产品会很擅长在工作中学习技能，再把它 all day I think the models and the products will get really good at learning skills on the job and then encoding 437 00:39:59,410 --> 00:40:07,300 写成文件、指令和技能，把原始的智能变得真正有用， that into files and instructions skills and taking that raw intelligence and actually making it useful for whatever 438 00:40:07,860 --> 00:40:14,080 用在你让它做的任何任务上。个人和公司也会更擅长 whatever task you're asking it to do And people and companies are gonna get much better 439 00:40:14,080 --> 00:40:19,360 把各种定制化流程是怎么运作的写下来，让它们 I think, about writing down how all of these kind of bespoke processes work so that they're 440 00:40:19,380 --> 00:40:24,180 能被智能体读懂和使用。接下来是一些有趣的问题，也许能启发 legible for agents to use So some Interesting problems I think that like could just spark 441 00:40:25,060 --> 00:40:27,360 你们去研究或探索 ideas for you all to research or to look into 442 00:40:27,700 --> 00:40:31,100 第一个，回到能运行一个月的智能体这个想法 The first one is like going back to this idea of agents that run for a month 443 00:40:31,100 --> 00:40:36,880 如果一个智能体要跑一个月，但模型每个月都发布新版，你怎么评测它？ What do you do when you need to evaluate an agent that runs for a month but the models ship every month? 444 00:40:37,320 --> 00:40:44,660 你没办法完全验证模型的表现是否符合预期 You know it's like you can't fully evaluate that the model is sufficiently behaving as expected 445 00:40:44,990 --> 00:40:49,130 所以要么想出更好的评测方法，要么别那么快发布模型 so you either have to figure out how to do a better eval or you know not ship models as fast 446 00:40:49,510 --> 00:40:54,030 我认为这是业界未来六个月必须解决的问题 And I think that's gonna be a problem that the industry has to figure out in the next six months 447 00:40:55,070 --> 00:40:57,310 记忆，当然，我们讲过各种算法 Memory, obviously, we talked about the different algorithms 448 00:40:57,670 --> 00:41:06,670 安全方面还有很多事要做：怎么做安全的框架，怎么让这些智能体周围的基础设施非常安全 I think there's so much to do on the security side for how to make a secure harness how to make the infrastructure around these agents very secure 449 00:41:07,270 --> 00:41:09,650 不让坏人把它们劫持 and so they can't be taken over by bad actors 450 00:41:10,730 --> 00:41:13,850 还有一点我们没怎么讨论 I think we didn't really talk about it too much but 451 00:41:13,850 --> 00:41:17,730 就是判断智能体什么时候该安静、什么时候该打扰你的经验法则 The heuristics around when the agent is silent or when it kind of bugs you 452 00:41:18,890 --> 00:41:25,340 我觉得现在还做得不完美，要在“主动”和“烦人”之间找到平衡，比如 I don't think that we have it perfect yet and I think trying to thread the needle between being proactive and suggesting like 453 00:41:25,400 --> 00:41:30,820 “嘿，我看到你昨天没去上课，我那里有录音，” oh, hey, like, I saw that you missed class yesterday like, I, I had a recording there 454 00:41:30,840 --> 00:41:34,360 “我记了笔记，给你”——这也许有用，也许有点烦 and I took the notes and like, here's the thing, like, maybe that's helpful, or maybe that's kind of annoying 455 00:41:34,620 --> 00:41:37,560 所以要想办法找到合适的 you know, and like trying to figure out the right the right balance there 456 00:41:39,520 --> 00:41:46,610 平衡点。我还觉得可以放心押注：明年这个时候，世界上流动的 I also think that it's a pretty safe bet to assume that at this time next year there will be significantly 457 00:41:46,780 --> 00:41:52,430 token 会多得多，这意味着优化推理、 significantly more tokens flowing in the world which means that optimizing the inference 458 00:41:52,490 --> 00:42:00,050 优化模型、优化框架，努力减少 token 用量、提高效率 optimizing the models, optimizing the harnesses trying to drive down the tokens to be more efficient will also 459 00:42:00,550 --> 00:42:03,530 让 token 用得更高效，也会变得非常重要 the token usage to be more efficient I think will also be very important 460 00:42:04,330 --> 00:42:07,010 所以有一些新的“积木”值得思考 So there's kind of these new building blocks to think about 461 00:42:07,010 --> 00:42:11,760 我记得是 OpenRouter 的 Alex，我在 I believe it was Alex from OpenRouter that I saw on on 462 00:42:11,760 --> 00:42:16,320 X 上看到他一个很好的观点。有人说，现在每个应用看起来都一样 X. He had a really good take on this There was people saying like oh, every app today, it looks the same 463 00:42:16,360 --> 00:42:21,140 有侧边栏，有智能体、有聊天框，大家都在做同样的东西 You've got the sidebar you know, you've got the agent when you got the chat box like everybody's building the same thing 464 00:42:21,600 --> 00:42:25,540 他说：对啊，就像 2005 年，大家也都在做同样的东西 And he was like yeah, that's like in 2005 he said, everybody's building the same thing 465 00:42:25,560 --> 00:42:30,820 有个服务器，有个数据库，有个 CRUD 界面来 They've got a server they've got a database they've got some like CRUD interface to like 466 00:42:30,920 --> 00:42:36,390 更新和删除数据。现实是，这就是新常态： update and delete things And the reality is this is just kind of the new normal now where 467 00:42:36,450 --> 00:42:43,670 每个新产品都得有某种智能体的一面，要么集成智能体，要么成为 every new product is going to have to have some kind of agentic side to it either integrating or it will become a 468 00:42:44,950 --> 00:42:50,670 供其他智能体使用的“无界面”数据。我们今天讲了很多这类产品： you know a headless piece of data for another agent to use and so we've talked about a lot of these products today 469 00:42:50,770 --> 00:42:57,720 持久工作流、沙箱、云端电脑、记忆、技能、连接其他服务、 durable workflows, sandboxes, computers in the cloud memory skills, connecting to other services 470 00:42:57,860 --> 00:43:01,760 语音、支付、身份、评测、可观测性 doing voice Payments, identity, evals observability 471 00:43:01,760 --> 00:43:07,680 每一块都可能吸引几十亿美元的风投，甚至上万亿 Each one of those boxes is like multiple billions of dollars of VC capital probably trillions 472 00:43:07,700 --> 00:43:12,220 每一块都有 20 家初创公司，每一块里都有太多值得探索的东西 There's 20 startups for each one of those And there's so many different things to explore in all 473 00:43:12,260 --> 00:43:16,380 因为它们是下一代公司的新基础组件 of those because they kind of are the new primitives for the next generation of companies 474 00:43:16,700 --> 00:43:21,340 是下一代产品的基础。它们都要集成进框架、成为其一部分。所以 the next generation of products And they all have to integrate and be part of harnesses. So there's 475 00:43:21,740 --> 00:43:25,740 这里肯定有很多事可做。最后几点要点，然后留时间 there's definitely a lot to do there A few things to take away and then we'll have time 476 00:43:25,740 --> 00:43:29,850 给大家提问。你们都是工程师，在思考如何写软件 for questions. Obviously you all are engineers you're thinking about how to write software 477 00:43:30,210 --> 00:43:34,710 现在比以往任何时候都更要思考整体系统设计和 It's more important than ever to think about the broader system design and the architecture 478 00:43:35,070 --> 00:43:40,050 代码架构，因为你在这里做的决定影响会不断叠加 of the code that you're working on because that the decisions you make there will really compound 479 00:43:40,750 --> 00:43:43,670 所以我仍然支持去看代码 So I'm still, I'm still pro looking at code You know 480 00:43:43,670 --> 00:43:47,370 我还是会看自己写的代码。有些人已经不看了，但我觉得看代码有帮助 I still look at the code that I'm writing. Some people are not looking at the code. I still think it's helpful 481 00:43:47,830 --> 00:43:49,690 尤其是为了理解你发布的东西的架构 especially to understand the architecture that you're shipping 482 00:43:50,230 --> 00:43:52,650 短短几年变化这么大，真是疯狂 It's crazy how much that's changed in just a couple years 483 00:43:53,470 --> 00:43:58,160 第二，听说你们上周在尝试自己搭建版本，或者理解 Secondly, you know it sounds like you all were kind of messing around with building own versions or understanding 484 00:43:58,480 --> 00:44:05,020 它是怎么运作的。我鼓励你们想想我今天讲的内容 of how the Worked in last week I encourage you to think about some of the things that I 485 00:44:05,060 --> 00:44:10,480 试着应用起来，想想怎么为自己做一个版本 did today and try to apply that and think about how you might build a version for yourself 486 00:44:10,480 --> 00:44:17,630 我很确定 GitHub 上有个开源版 Grok Bot，有人 fork 了我们的代码 I'm pretty sure there's like, open Grokbot on GitHub somewhere where somebody's like, you know, slot forked, our code and made 487 00:44:17,910 --> 00:44:25,810 做了一个版本，你大概可以给它搭个图形界面，配个沙箱 and made a version and you could probably set up like, the, the GUI for that with you know, a sandbox somewhere and 488 00:44:26,330 --> 00:44:31,220 再配台虚拟机之类的，挺有意思的，如果你想 you know, a, a VM and, like all sorts of different things so that could be interesting if you want 489 00:44:31,240 --> 00:44:33,260 多了解这些部件是怎么运作的 to learn a little bit more about how these pieces work 490 00:44:33,880 --> 00:44:39,480 最后，别害怕多学一点大语言模型的原理 And then finally I would not shy away from just learning a little bit more about how LLMs work 491 00:44:39,700 --> 00:44:45,000 今天只简单讲了它们怎么训练，但还有很多可以 You know we gave a little bit of the TLDR here of how they're trained but there's so much more you can go 492 00:44:45,100 --> 00:44:52,000 深入的，不一定要钻研机器学习或科学部分。但理解它很有帮助 into here without necessarily going deep in ML or deep into the science part of it. But I think it's helpful to understand 493 00:44:52,340 --> 00:44:57,520 就像做网页应用时懂数据库原理很有用一样，因为越来越多 just like when you're building a web app it's helpful to know how databases work because increasingly the 494 00:44:58,240 --> 00:45:04,160 你在产品和框架层面的决定，都受到模型本身的影响 there are so many decisions you make at the product and the harness level that are influenced by the models themselves 495 00:45:04,400 --> 00:45:08,640 对它的运作有直觉，会让你在 and having an intuition of how that works I think will make you very well positioned for 496 00:45:09,120 --> 00:45:13,200 无论什么岗位上都很有优势。我今天就讲到这里 whatever role you're operating in And, that's all I have today 497 00:45:14,160 --> 00:45:15,160 谢谢大家 Thank you