山姆·奥尔特曼：我为什么反对单一的超级智能

咱们不按台本来了。就是喝杯啤酒。这不是第一回不喝健力士（Guinness）了，但确实是第一回喝IPA。所以嘛，啥都是头一回。干杯。干杯。We're going off script. We're just having a beer. It's not the first non-Guinness, but it is the first IPA. And so, it's the first time for everything. Cheers. Cheers.

是花了大概二十年呢，还是就花了最近这几个月？Why did it take like the 20 years or why did it take the last few months?

问题是，虽然咱们早就知道，这些模型越来越擅长在实战里找漏洞。第一，它们进步挺快；第二，大家还没真拿它们去全网扫一遍。网上的代码和暴露面太大了。现在大家开始这么干了，我觉得我们会发现，实际存在的严重漏洞比预想的要多得多。就算一直盯着这行的人，心理预期已经很高了。So the problem is that although we have known for a while that these models are getting good at finding vulnerabilities in the wild,A, they continue to get better fairly rapidly, and B, people just hadn't really used them to go look at everything out there. There's like a lot of surface area and a lot of code on the internet. And as people have started to do that, I think we're gonna find that we are in a world of many more serious vulnerabilities than we expect, even though many people who have been following this for a while expect a lot.

我觉得确实该这么做。对。听着挺难，可咱们现在有特别棒的新技术来搞定它。所以我猜这事儿迟早会成。你看，已经有一批新创公司开始聊了。他们想为这个AI越来越聪明的世界，搭建全新的互联网或应用架构。Like that seems to me like what needs to happen. Yeah. It would seem really hard, except we have this amazing new tool to do it with. So I kind of think that's what's gonna happen. Yeah. And you're already seeing like a new generation of startups talking about building this like new kind of internet or new kind of application architecture for this world where you assume that the models get smarter and smarter.

我的意思是，大家越来越爱聊一个话题：咱们得重新设计路由器、操作系统甚至应用的底层架构。你知道的，我看到有些新创公司说，打算直接搞一套全新技术栈。要是两年前听这话，肯定觉得疯了。你绝对会说，不可能啊，体量太大了，根本写不完。I mean, people are talking more and more about just like, let's re-architect what it means to be a router or an operating system or an application. And you know, I've seen some of these new startups saying that they're just gonna like make a whole new stack. And that would've seemed insane two years ago. You would've been like, there's no way, there's too much, you can't write all that.

也许很快你就能做到了。我敢打赌，咱们搭建系统和网络的方式，会迎来全新的模式。And now maybe soon you can. I would bet that we have a completely new sort of model of how we make systems and networks.

最让我想起早期互联网的，是那种难以置信的创造力。大家用起这些工具来，简直毫无束缚。尤其在孩子身上特别明显。我记得第一次用上带网络的电脑时，能做很多大人根本想不到的事。看着八九岁的孩子，用些大人想不到的疯狂点子，做出完整的游戏。他们理所当然地认为，技术就该这样伴随一生，而且越来越好。心里总想着，我要做个有用的东西给朋友看看。小时候咱们做软件，受尽折磨，结果功能还寥寥无几。现在十分钟就能搞定一个完整的三维游戏。太不可思议了。你脑子里的任何想法，十分钟都能实现。这真像早期的互联网。还有这些人，他们永远不会为此感到震惊。因为他们对世界的期待就是这样。不管有什么点子，想创造什么，觉得有用的，或者能改善生活的，都该由电脑来实现。OpenAI（OpenAI）有次出了严重故障，代码模型(Codex)直接挂了。The thing that most reminded me of the early internet was, you know, there is this like spirit of just like unbelievable creative unleashin people using these tools. You see it in kids particularly. And again, maybe this is just like, 'cause I remember what it was like when I first got a computer with internet and I could like do all these things that adults just couldn't even think to ask for. And like watching a, you know, eight or 10 year old make a complete video game with some crazy ideas they have that like no adult would really have and expect that to just be the way that technology is gonna work for the rest of their lives. In fact, getting better and better. And just like this constant, I'm gonna make useful stuff for me and show it to my friends. Like we had to suffer when I was a kid to make a piece of software that barely did anything. And now you can have like a full 3D video game, like unbelievable, anything you ever think of in 10 minutes. That reminds me of the early internet, including the just like these people. They're never gonna be mind blown by that because that's always what they expect out of the world. Like any,any ideas they can come up with, anything they wanna create, anything that they think might be useful, anything that they just need to make their lives better. Like the computer's supposed to make it. There was a crazy moment at OpenAI where we had a sev and Codex went down.

倒不是完全忘光，但那种感觉就像在问，万一不会了呢。我读书那会儿，常跟老一辈程序员聊天。他们偶尔会掏出巨大的施乐 Alto（Xerox Alto）软盘。或者展示一行行手写的汇编代码。甚至翻出打孔卡。我觉得特别不可思议，大家居然靠这种方式，一步步把我们带到了今天。I mean, not all the way, but it was like a whoa, what if you couldn't. I remember when I was a student and I would like meet the really old guysand they would like once in a while, you know, pull out their gigantic Xerox Alto disk or they would like pull out their like, you know, with like line by line written of assembly or they would pull out their punch cards. And I felt like it is incredible to me that you all bootstrapped us to this point doing it this way.

那个段子好笑是因为太真实了。幽默得扎根于现实，点出社会的荒谬之处。说到网络安全，大家会问，谁要求这个的？你们为啥要做？这是我的一大失望，也是对我们领域的批评。模型擅长网安就是例子。我们没把技术潜在的风险值不值得承担解释清楚。人们觉得我们在画大饼。你说模型能治癌症，也许吧，挺好。你说能降价，也行，还能看病咨询。但接着你描绘的未来像新一轮工业革命。上次革命未必多好，这次听起来更糟。我会失业，失去对未来的掌控和自由。人工智能太危险，得靠极权政府才能保命。Look, the reason that SNL thing was so funny is because it was so true. Like, humor like that doesn't stick unless there's an element of, like, it's like clearly pointing out a ridiculous but true thing about society or someone.And the cyber thing,there is a point which is like, okay, you know, sure, the models are really good at cyber, but like, who asked for this? Why are you doing this? Like, one of my big disappointments, criticisms of our field or whatever, and I think maybe the cyber, the model's getting good at cyber is an example of this, is I don't think we have done a great job of explaining to people why we think all of the potential downside of this technology is worth it. Like, the kind of caricature of this is people feel like we're like, all right, you know, fine, you say the model's gonna cure cancer someday, maybe it will, and if so, I guess that's good. And you say it's gonna like make stuff cheaper.And okay, maybe I already see that's good and I get access to medical advice or legal advice. But then you paint this picture of the future where it's like a new industrial revolution, and maybe the last one wasn't that good and this one sounds worse, and I will have no job, no ability to like have control or choice or freedom of my future, and AI is so dangerous that you're gonna have to like have a crazy state so we don't all die.

这简直是个糟糕的销售话术。是的。我对开放人工智能（OpenAI）最骄傲的是，即便在安全圈里，我们推动迭代部署、把工具交给大众很不讨喜，但我们始终坚持这个原则。我认为这才是正路。光告诉用户模型能帮他们干大事还不够。我们要让用户过上自己定义的好日子，去干有用的事。你不能光坐着领低保，然后让旧金山的一帮人替你决定未来。This is like a terrible sales pitch. Yeah. And one of the things that I am most proud of about OpenAI is even when it has been hugely unpopular in the AI safety community for us to push for iterative deployment and putting these tools in the hands of people,we have really stuck to that principle. And I think this is the only kind of good path. I think people, it's not enough to tell people like, you're gonna get the model to do these great things for you. It's gotta be like, we are going to enable you to have a better life that you define for yourself to do useful things. You're not gonna just sit and, you know, get a basic income and have a bunch of people sitting in a few square miles of San Francisco make all the decisions about the future.

对，当然不是什么都往外推。上线前还得确保它们安全，不能在社会上乱来。但要警惕能力过度积累，这点我们必须严防死守。Yeah, I mean, not all, like, you still have to make them like safe before you put them out there. You can't like cause havoc in the world. But the building up a capability overhang is a thing to fight against.

方向上确实如此。没错，我们要把技术推向世界，交到用户手里，让大家用它去探索各种奇妙可能。That directionally speaking, yes, you wanna get this into the world, you wanna get this into people's hands, and you wanna let people use the technology to figure out all these wonderful things.

关于PHP漏洞的事，我绝不轻言胜利。毕竟我们还不确定全貌，预料中肯定还会出点状况。回想十天前刚上线时，其实比我最坏的打算好多了。我们听到不少反馈，大家不仅用我们的模型打补丁、做防御，还有公司直接说：‘我就快速扫描一小时，先把服务停掉，用完再恢复，反正应用本来就在离线状态。’现在想想我这话可能有点乌鸦嘴，但第一天确实比我预想顺利。很多人已经用上它了。不过很多网络安全专家最近都在讨论一件事：他们不再确信我们的威胁模型假设是对的。他们说：‘如果一切如我们所想，硬件漏洞挖掘像我们认为的那样难，那这方案行得通。但如果AI能发现极其隐蔽的新型硬件漏洞，涉及我们不懂的物理机制，导致原有假设全崩，那接下来会发生什么，大家心里都没底。’I don't wanna declare victory at all on the PHP thing, 'cause I still don't think we know. And I assume some bad things are going to happen. Relative to my worst fears, if I can like think back 10 days ago, the first day of that went much better than I expected.And we heard a bunch of stories from people that were using our models to, I mean, not only build patches and build cyber defense, but like individual companies who were just like, "I need to just do a quick scan appointment. Like, I'm gonna take my thing down for an hour. I'm gonna use this thing and I wanna put it back up 'cause my application's offline." Now I feel like I really jinxed it and I shouldn't have said that, but the first day did go a lot better than I thought. And I think people have been able to use that.The thing that I've heard a lot of cyber experts talk about isthey just no longer are confident that we have the right assumptions baked into our threat model. And they're like, "Look, if everything is what we think and hardware exploits are as hard as we think they are, then yes, we can see how this is gonna work. But if AI is going to discover really deep novel bugs in hardware that require physics we didn't understand, but all of a sudden means some assumption doesn't hold, then I think people aren't sure what happens."

我的直觉是，首先，我们本应该比现在更清楚怎么做，也确实该做到。眼下最核心的任务，肯定是升级安全模型。你想象一下，如果一个理论上的AI，比最顶尖的安全研究员聪明一百万倍，你还真敢信任那个沙盒吗？My instinct, well, first of all, I think we knew much better than we're doing now, and we should. I think we clearly, that is the main thing to focus on right now is a better security model. If you imagine a theoretical AI that is like a million times smarter than the smartest security researcher, would you like really be willing to trust that sandbox?

尤其是模型能发现人类还不了解的新科学。我们可以用模型来搭建沙盒，我觉得可以。好，这点容我稍后回来谈。我认为沙盒隔离有一定作用。监控也能更进一步发挥作用，让我们能看到模型在做什么。不仅看它的思维链，还能做激活监测。真正去弄明白：模型有没有在欺骗我们？我们对此满意吗？最终，你必须高度确信模型是真正对齐的。它们会珍视并努力遵循操作者的真实意图，并且希望人类永远掌控未来。只要你对模型的对齐性绝对放心，就能用它来不断升级沙盒。但如果你真的把目光投向未来，想象一下拥有比我们聪明一百万倍的东西意味着什么？我不知道。那听起来会很奇怪。因为对齐会变得很难。因为你必须极度信任对齐性，才能相信这些沙盒能始终领先于能力发展。especially if the model can discover new science that people don't already understand. You can use the models to construct the sandboxes. I think you can. Well, so this gets, let me come back to that.I think sandboxing works to a point.Then I think monitoring works to a further point, where we can look at what the models are doing, not only in their chain of thought, but we can do activation monitoring and try to really understand, are the models, are we happy that the models are not deceiving us? And then eventually, I think you need to believe with huge confidence that the models are truly aligned and are going to value and try faithfully to do, to follow their actual,the intent of the people operating them, and that they're going to value sort of people being in control of the future forever. And if you had a model that you could fully trust the alignment, that you were absolutely certain of the alignment of, then yes, I think you can have that to help you build better and better sandboxes. But if you really let yourself project forward into the, what does it mean to have something a million times smarter than we are?I don't know. I think that just gets strange. Because the alignment becomes hard. Because you need to really, really trust the alignment to trust that these sandboxes stay ahead of capabilities.

我觉得这是个很有前景的方向。我琢磨着这个思路……说实话我不爱听“智能体集群”这词，咱们叫“智能体团队”吧。不过这说法好像越来越火了。我推测，智能体团队会有很有趣的缩放规律特性，在对齐方面也很值得研究。我觉得咱们应该多往这方向推，不光为了提能力，安全层面也得重视。I think this is a promising direction.I suspect that this idea, I hate the word agent swarms. Let's call them agent teams. But it seems to be taking over. I suspect that teams of agents have very interesting scaling law properties, but also very interesting alignment properties.And I think that's a direction we should push in more, not just from the capabilities, but from a safety perspective too.

程度确实不一样。看这个问题有个角度：我们到底能往单个任务上投入多少算力来真正帮到忙？几年前这事发展太快，大家可能忘了。那时候你往一个问题里投超过一美元的算力，基本没用。为啥？因为GPT-4顶多给你磨个十万个词，回答个问题还行。但它绝对写不出实用的软件。咱们当时还没摸清怎么用测试时算力来提升结果。后来搞出了思维链，模型能深度思考了。大家才开始大量用算力。现在你能派一整支团队去干活，还能并行一万次。Not to the same degree. Like the one way to look at this problem or this question is how much compute can we productively throw at a single problem? And you know, a few years ago, we forget how fast this all got, but a few years ago it was like impossible to throw more than a dollar of compute at a problem and have it help you at all. Why?You could have GPT-4 like grind out for you, you know, like 100,000 tokens or something, and it was gonna answer your question. And it certainly was not gonna be able to go write productively, like write you a piece of software.We just, we had not figured out how to use a lot of test time compute to get you a better result. And then we figured out chain of thought and the models could think hard about a problem. And now, and then like people really started using a lot of compute. And now you can throw a whole team at something and you can do that, you know, 10,000 times in parallel.

但这带来的影响，我觉得全世界都没太关注。其实有两点很关键。第一，单任务可投入的算力呈指数级增长，结果越来越好。第二，普通人用AI的方式，和专业人士高效用AI的方式，差距正在越拉越大。But so the effect of this, I think this is not being well followed, paid attention to in the world, but the incredible exponential of how much compute you can throw at a single problem and get better results is one thing. The delta between like what most people are using AI for and what the people are using AI the most productively for has just continued to spread and spread and spread.

其实可以啊。哦，明白了。你可以直接说：在一万美金的算力预算内，把活儿干到极致。You can. Oh, okay. Well, do a good job. Yeah, if you say do the best job you can with $10,000 of compute.

对，它能听懂这个意思。Like, yes, it will understand that.

我的意思是，社会还得面对另一个难题。两家公司竞争，一家说一万美金能解决，另一家说十万美金。结果显而易见。但这引出了个问题：社会该怎么看待算力分配和财富的关系？不过我们在OpenAI（OpenAI）内部也有自己的做法。我们把大量算力投入内部提效。只要大家不滥用，公司会给足自由度，让大家自己拿主意。I mean, there's like a, there's another kind of like hard question for society about what does it mean when you have two companies competing and one says you can spend 10,000 and one says you can spend 100,000 on the same problem. Like, it's clear what's gonna happen there, but there's like a question about how society thinks about compute allocation and wealth and things like that. But we have our own version of this inside of OpenAI. We allocate, you know, a lot of compute to internal productivity and then people get very wide latitude to make their own decisions unless they're really being abusive.

没错。我们尽量不奖励消耗大量算力的行为。比如我们不搞令牌排行榜。在排行榜上靠前可不是好事，尤其影响晋升。我们的默认态度是保持怀疑。但只要你证明自己在做重要的事，公司只会全力支持你。Correct. We try never to reward people for using a lot of compute. Sure. Like no token leaderboards. It's actually slightly bad to be higher on the token leaderboard, certainly on a promotion. But it's like, it's a skeptical default and if you can show that you're doing something important, you will have nothing but appreciation.

说实话，在数学项目上，我们确实故意让它失控过一周。因为你不试永远不知道边界在哪。而且这招还真奏效了。新范式出现时，必须给团队充分的自由。比如我们之前尝试证明纳维-斯托克斯（Navier-Stokes）方程。当时完全不懂多智能体团队的扩展规律。到底该用最强模型？还是多复制几个？长序列还是短序列？我们一无所知。现在懂了一点，还能更好。但就像所有新领域一样，我相信每年都能把资金或浮点运算次数的使用效率提升十倍。我们公司特别擅长优化资源分配，这也符合这个行业的重要特质：让人去探索。Look, we did let it get out of control for like one week on the math. Intentionally so, because you have to be like, and it worked. Well, you don't know how, you don't know how far it's gonna go until you try. Yeah, yeah. And I think it's important to have, when there's a new paradigm, I think it's important to let people be fairly unconstrained. So when we did the Navier-Stokes proof, we understood absolutely nothing about the scaling laws of teams of agents. You know, are you supposed to like-- The very recent Navier-Stokes proof.Yes. Yeah.Are you supposed to give them like the smartest model? Are you supposed to give them like many copies, fewer copies, longer rollouts, shorter rollouts? We did not understand that at all. And now we understand it a little bit. I think we can still understand it like much better. But like has happened in every other new paradigm we've discovered, I am confident that we will find our, you know, 10x improvements per year at how we allocate dollars or computer flops, whatever you wanna measure, into useful things. So I think this is a thing that our company is particularly good at, but also like fits an important property of the field, which is let people explore.

在可能性的极限处，然后相信强大的优化压力会找出高效的方法。方向上来说，我更愿意让大家放手使用海量算力。让高端用户去探索能力边界，然后再想办法大规模提效。at the limit of what is possible, and then trust that the incredible optimization pressure will figure out how to do this in an efficient way. I, and, you know, directionally speaking, I would rather us be more willing to let people use crazy amounts of compute, and let users have extremely high-end plans to figure out the edge of the possible, and then figure out how to make that efficient at a mass scale.

我觉得我们已经在做了。我敢打赌，明年重做纳维-斯托克斯方程的研究，成本能降到现在的百分之一。当时我们大概花了……Well, I think we're already doing it. I mean, like, I,I bet we could redo Navier-Stokes at one one-hundredth of the cost a year from when it was done. So we did it in like--

对。模型更聪明，方法也更优。数学研究大概要花多少钱呢？像纳维-斯托克斯方程，我想应该在一千万左右。好吧，可能再多一点。Yes. Yeah. Smarter models and better approaches. How much shall the math cost? Order 10 million per, like, for, like, Navier-Stokes, I think. Okay. Maybe a little more.

不不，因为后来我们进步太快了。理解也越来越深。就像——我明白了。懂了。对，平均下来，每项花的钱远少于那个数。Well, no, no, 'cause then we kept getting better so fast. Like, we understood more. Like-- I see. Got it. Yeah, we spent way less than that on the average one of those.

对对对，就是这类工作。有意思。但这笔钱其实包含了比我们已发表内容多得多的其他发现。是的。所以算下来，可能确实花了那么多。Yeah, yeah, yeah, that kind of work. Okay, interesting. But that include, like, that includes, like, much more other discovery than we, what we have published. Yes. So it was like, maybe we spent that much.

我们正在摸索怎么平衡，这确实是个难题。一方面非常明确，我对此毫无保留：人类理应获得基础科学知识。我们不能把重大发现藏着掖着。世界有权利知晓这些。我认为这比另一项重要权益更优先，那就是科学家。如果从事科研，你的职业生涯和生活中的大量成就感都来自发现。所以如果我们打算继续做数学，或拓展到其他领域，我觉得我们理应为从业者想办法。我们要找到一种方案，既能让人类受益，也能让他们拥有最好的职业未来。所以我们现在正全力解决这个矛盾。We are trying to learn how to, this is like a real tension. I think on one hand, very clearly, and I feel unequivocal on this point, humanity deserves fundamental knowledge, and we should not be sitting on a major fundamental science discovery.Like, the world deserves that and gets to know it. And I think that outweighs another very important equity, which isif you are a scientist, your career, and, you know, a significant amount of your enjoyment of fulfillment in life comes from discovery in science. And so if we're going to continue to do this for math, or begin to do this for other fields, then I think we more than owe it to the people that work in those fields to figure out, you know, what is the solution space where the world gets benefit from the knowledge, and they also get the best version of the future that they can have. So we are working through that right now.

就像我刚才说的，我觉得这对AI没坏处。从整体看，世界有权了解宇宙怎么运转。但我确实能理解数学界的心情。我觉得这事非做不可。比如要是能搞懂物理学的终极理论，或者治好一堆病，那肯定是世界该知道的知识。但我想找条路，既能推进研究，又尽量体谅那些几百年来推动这行的社区。这确实挺难的。As I said, I don't think it was bad for AI. And I think on net, the world deserves to understand how the universe works. But I do have empathy for the math. Like, I think we need to do this. I think we should do this. I think like if we can go understand how to, you know,a theory of all of physics or how to cure a bunch of diseases, like clearly that is knowledge the world should have.But I would like to figure out a way to do thisthat is as sensitive to the communities that have driven these fields forward for centuries as we can. And it's just a, it's a hard thing.

但你承认数学家们确实不太高兴。But you agree the mathematicians are not that happy.

不不不不不，我是这么觉得的。我为啥会有不同感受呢？有个周末我们上线了新模型。其实这话我不该说，但既然咱们在喝酒，我就直说了。没错，咱们已经喝了半品脱了，对吧。这时候聊正合适。那个频道叫“模型名-拿住我的啤酒”。人们就把最难的问题丢给它。大家本来都不指望模型真能搞定这些。但我们集体见证了学术生涯里最难的科研难题，被模型接连攻克，横跨好几个领域。看着太震撼了。连续两个晚上，我几乎通宵没睡，兴奋得睡不着。好多问题后来都发表了，还有其他成果。我当时特别激动开心。但我心里也多了些难过和惆怅，有种某种东西正在结束的感觉，比我预期的强烈得多。所以再说一遍，我同意这对世界是对的。我也同意这就是现实的样子，没人能拦着，也没必要拦。但我确实有那种真实的……No, no, no, no, no, I do. Why do I feel differently about this? There was one weekend when we got a new model and in, I'm actually probably not supposed to say this, but I will since we're drinking. Exactly, we're half pints in, yeah. Perfect time for this. The name of the channel was model name dash hold my beer.And people were just throwing that at the hardest problems they could.And people just like didn't think the model was gonna do these things. And we had this collective experience of people's like hardest research problems from their academic career. The model was just knocking down one after the other, many different fields. It was incredible to watch. And kind of two nights in a row, I was up most of the night 'cause I was so excited I couldn't go to bed. It was like, it was many of those problems that we published, many other things. And I was tremendously excited and happy.But I also had like much more sadness and melancholy and like a feeling of like something ending than I was expecting. So again, I agree this is the right thing on behalf of the world. And also I agree that this is just like the shape of reality and nothing that anyone can stop or should stop. But I do have like real.

这关乎人们的生活，也给他们带来巨大快乐。说实话，绝大多数数学家一辈子也解不开千禧年大奖难题（Millennium Prize Problem）。但那种“也许我能解开”的念头很动人。世上还有太多未知，只要肯下功夫，放下其他事全心投入，依然能破解和理解。这本身就很美。从安全角度看，你还期待什么？我们之前聊过一点，我想再强调下。我先叫它群体智能（Swarms）吧，这词现在挺流行。我觉得群体的特性，很像社会韧性或社会稳健性的特质。我觉得我们现在缺的就是这种。尽管社会毛病不少，但它极其坚韧。这全靠很多人互相制衡、协作，还有多元和民主的价值观。我之前有点伤感，觉得人工智能（AI）发展会背离这个方向。但现在我很期待这条路线。It's like it's people's lives and it's something that has brought them great joy. And, you know, maybe most, I mean, certainly most mathematicians never solve the Millennium Prize problem, but the idea that you might be able to, and the idea that there is stuff out there that we don't understand yet, that through great effort and, you know, pushing aside other things in your life and dedicating yourself to, you can still solve and understand. Like, there's something beautiful about that.What else are you excited about from a safety point of view? We talked about this a little bit earlier, but I wanna like emphasize it again. I'll just call it swarms since that seems like the term right now. I do think the properties of swarms match a lot of the properties of the resilience of society or the robustness of society that I kind of, I think we're just missing from the like single rollout, single-brained version of the world.Like, for all of the deep flaws of society, it is unbelievably robust.And this would not happen without like a lot of people keeping each other in check and collaborating and kind of these values of pluralism and democracy. And I had been sort of sad that that was gonna trend away in AI. And now I like, I'm excited about this as a direction.

直觉上我更希望，有一万亿个略超人类智商的人工智能（AI）加入社会，而不是只有一个智商高达一万亿的AI加入。I would much rather intuitively havea trillion AIs of like slightly superhuman intelligence join society than one AI of 1 trillion IQ join society.

这些都是我们必须推动的方向。我们有这项了不起的技术，它可以指向不同的用途，以多种方式落地。很多人对它的去向、用法以及理想世界的模样都有不同看法，但是……All of these things are things we have to push for. Like all of these things are like, we have this incredible technology.It can be pointed and deployed in many ways. I think there are a lot of people who may have a different opinion about where it should, how it should be used and what is the best version of the world, but.

我觉得这才是对的方向。This is the one that I think is right.

我的意思是，曲线发展过程中总会有些失控的时候。任何重要的新事物都会经历非理性狂热、泡沫破裂、限制过剩和起伏波动。但我认为趋势是向前的，我们永远需要更多算力。只要价格合理，够用的模型需求只会涨不会跌。我们正做很多准备。我们造了一款芯片，在单位能耗下的智能表现惊人。我们正在学习自建数据中心。我们还启动了机器人项目，打算用它来大量建造数据中心。大家总爱讨论……I mean, there are all these ways where I think thingsat various times along the curve are gonna get somewhat out of hand and there will be moment, like with any new thing of this importance, there are going to be moments of irrational exuberance, there are gonna be moments of bust, there are gonna be moments of constraint and glut and up and down. But I think the trendis toward a, we always just want more compute. Like the demand for sufficiently capable models at a sufficiently low price, I think that trend only goes one way. And we're doing a lot of things to try to be prepared for this.We made a chip that is an incredible performance at like, you know, units of intelligence, performer at units of intelligence per watt. We are learning how to build our own data centers. We have a robotics program that we're gonna point at making lots of data centers. People talk a lot about-

对，大家老是把RSI当成搞AI研究的超级智能。但很少人聊自复制的数据中心。你想啊，用AI控制机器人，运营晶圆厂，制造涡轮机或太阳能板，然后盖数据中心。我觉得这已经触手可及了。所以我们会拼命搞出超便宜的算力。Yeah, people talk a lot about RSI as like a AI that does AI research. There's very little, relatively little talk about like self-replicating data centers. You know, this idea that you can use AI to control robots and run fabs and build turbines or solar panels and, you know, build data centers. But I think we are, I think we have that like within grasp.But, and so we'll try to make a lot of really cheap compute.

难道零部件还能比机器人多吗？Can we have more components than robots?

迟早会有的。对对对。这一切就是我们要长期逐步解决的问题。Eventually. Yeah, yeah, yeah, yeah. And all of this is like how we solve this over a long period of time.

解决方案应该随时就会出来。目前进展怎么样？不太妙。我不是说这是OpenAI该管的事。我觉得这得靠市场那只看不见的手吧？就像内存价格涨得太离谱了。所以很快就会有新的内存厂商涌入市场。And the solutions should be here any moment now. How are we doing on the solutions front? Bad. And I'm not saying this is OpenAI's job. I feel like this is the invisible hands job, right? Like memory's gotten really expensive. And so new memory makers are gonna flood the market any moment now.

这就导致他们在资本开支上很保守。很明显。按理说，你应该看到供应链各环节都在动作。新创公司入场，或者更关键的是，有经验的人加入。比如英特尔就造芯片。他们大概率能重返内存领域，按惯例嘛。照旧行事。咱们回到正题。Which leads them to be conservative in their CapEx. Clearly. It feels like what you should expect to see is that every partof the supply chain, both new startups coming along, and maybe more relevantly, people with relevant experience getting into. So, you know, Intel makes chips. They can probably make, they can get back into the memory game as is tradition. As is tradition. Switch back.

涡轮机、电池，还有混凝土。Turbines, batteries, concrete.

主要是机器人的零部件。不是那种演示用的假人形机器人，而是能真正加快生产速度的工业机器人。Some components for robots. And not like the humanoid, like, you know, kind of like fake demos, but like the industrial robots that can just make stuff faster.

对。说实话，如果我们开放人工智能（OpenAI）开始卖自己的芯片，那就算一个例子。Yeah, honestly, I mean, maybe if OpenAI sells our chip, maybe that would be an example.

至于光刻（Lithography），地球上好像只有一家公司真懂怎么做。但芯片设计，也就是准备送交流片（Tape out）的文件，大家已经很熟了。这正是人工智能（AI）大展身手的绝佳领域。毕竟全球硅片产能有限，大家利益一致。我很自豪的是，开放人工智能（OpenAI）有一种文化：哪怕大家习惯老办法很久，也特别愿意尝试用人工智能（AI）把它做得更好。Well, in terms of like how to do the lithography, yeah, I think there's like one company on earth that seems to know how to do that. In terms of how to do the chip design, like to build the file that you eventually send to tape out, that is well understood and that is a great problem for AI. And given that the world is going to be like limited in its silicon capacity, we all, we have like a shared interest. One of the things that I'm proud of about OpenAI, and maybe it sounds obvious, but it was maybe like not as obvious at the time, is we have a culture where people, even if they've done something the old fashioned way for a long time, are super willing to like try using AI to do it better.

人工智能（AI）研究、软件工程。现在还包括采购，甚至一些大家觉得完全不在人工智能（AI）应用范围内的事。AI research, software engineering, but also increasingly like procurement or things that people think of like very outside the realm of what most people are using AI to do.

所以我认为我们还没完全摸清它的真实运作方式。目前我们只知道怎么提要求，比如“帮我提升百分之五的计算效率”。但这只是高层级的指令。就像我现在完全可以成为一个极其成功的人工智能（AI）研究员。So I think we don't know yet how it truly is gonna work. We know today how to say like, find me a 5% computer efficiency win. But it's that high level. Like I could be an incredibly successful AI researcher now.

我也不确定，太不可思议了。不过你要是找工作，欢迎来试试。现在大家还不会做的是：去发明一种全新架构。它的突破得跟Transformer（Transformer）一样大。我觉得这就是趋势。I don't know, incredible. But yeah, if you were looking for a new job, you can come do it. What I don't think anyone knows how to do yet is to say like, go discover a new architecture that is as big of a single leap forward as the transformer was. And I think that's where things are heading.

更聪明的模型。在这个维度上更聪明的模型。Smarter models, smarter models that are smarter in this dimension.

如果把让我们模型擅长数学的那套流程，用到AI研究上，应该能行。就像刚才聊的，短期内我们可能得给单模型的智能程度设限。但进度慢是因为这个决定，不是AI做不好研究。I think if you apply the same pipeline that we applied to get our models to be so good at math to AI research, like it will probably work. Now, per an early part of this conversation, I kind of think that at least for a while, maybe for a long while, we are going to have to agree to some limits about how smart we let these single models go. But a lack of progress will come from that decision, not from the AI's inability to do good research.

大家常聊OpenAI一个有趣话题，听着明显但算好答案：比如给我造个RSI。但问GPT-7时，光说这话够吗？肯定不够。光说提升百分之五算力也不够。这现在是热门争论。答案不只是问什么，还包括智能体集群（Agent Swarms）这类玩法。我们该怎么用这些模型，才能发挥最大价值？未来三个月我们会学到很多。One of the sort of interesting things that people talk a lot about OpenAI, which sounds very obvious, but it sounds like a good answer, just like, well, build me, build me RSI or whatever is, what are the questions that you're supposed to ask of GPT-7? Probably it's not enough to just say like, make me RSI. And also like, it's probably not enough to say like, buy me a 5% compute efficiency win.I think this is like one of the most interesting debates going on right now. And the answer is not just like what question to ask it, but it's things like agent swarms. Like what do we go, how are we gonna use these models in a way that like actually maximizes what we can get out of them? I think we're gonna learn a lot about that in the next like three months.

以前很多公司会在日历里加个功能，显示每个会议的大致成本，按参会人员的薪资算。我觉得那其实还是低估了。保罗·格雷厄姆（Paul Graham）说过一句话，很多人也认同：有些员工能省下的钱，可能还没一周办公室零食费多。但公司却开着毫无价值、烧钱惊人的会，而且没人能叫停。说实话，我这话有点偏袒我的书，但我真心觉得：跟其他开销比，AI使用根本不是公司最大的花费问题。就算它有点浪费，也不值得重点盯着。There was a time when a lot of companies had this like thing in their Google calendar, which showed the like approximate cost of each meetingin terms of like everybody's comp that had to be there. And I think that was always still an underestimate of it. And there was this observation that like Paul Graham made it at one point, you know, and many other people made it too, which is that there was like, there are people at companies that have incredible power to save like, you know, $17 on the snack budget of the office every week. And yet you can have these meetings for like unbelievable amounts of money that are adding no value and that there's kind of like no equivalent of the ability to stop that. I think that most use, okay, so I'm like really talking to my book here and I'm very biased, but I genuinely believe it. That use of AI is like not a company's biggest spend problem relative to other things that happen. And even if it's like somewhat inefficient, it is not where I would focus.

你想把Stripe（Stripe）的全部营收和利润都砸给OpenAI（OpenAI）服务，然后来抱怨这个吗？那我可太乐意跟你聊聊了。You wanna go send all of Stripe's revenue, all of Stripe's profit on OpenAI services and then complain about this? I'd be delighted to have a conversation.

你觉得合理的数字应该是多少？比如今天应该定在多少？Do you have an intuition on what the right number is? Like what it should be today?

我觉得对很多公司来说，答案显然是肯定的。而且我认为总有一天，大家会真正去优化这部分开支。另外，我为OpenAI感到骄傲的一点是，我们一直在拼命把单位智能的成本往下压。我觉得自己算是这方面的积极倡导者。不过即便如此，我认为当下的问题依然是，用AI太少的公司比用太多的公司要多得多。I think for many companies, the answer to that is clearly yes.And I think there will come a time whentheright thing to do is to really optimize that. And also, like, another thing I'm proud of about OpenAI is we've tried to really drive the cost per unit of intelligence down and down and down. I think we've been like a good spokesperson and good champion of this. But still, I think the problem, even at current costs, I think there are more companies the problem of not using enough AI rather than using too much AI.

应该会降。That should go down.

肯定会降。That should go down.

我的意思是，它之所以可能涨价，是因为AI每瓦特能提供的价值飙升得太快。加上供应严重短缺，大家愿意掏更多钱。I mean, the reason it could go up is that if AI just gets increases in value per what it can do per watt so quickly, and there's such a supply shortage that people are willing to spend much more.

没错，我就是觉得算力没跟上。如果看把沙子变成智能的基础成本，那肯定得降。能源成本要降，芯片成本要降，供应链其他环节的成本也得降。要是降不下来，我会很难过。但要想成本降下来，考虑到这东西价值这么大，供应量必须大幅上涨。But I would view that as compute not keeping up. And if I look at the like fundamental cost of turning the sand into intelligence, that should go down. The cost of energy should go down. The cost of chips should go down. The rest of the cost of the supply chain should go down. If it doesn't go down, I'll be very sad. But for it to go down, given how much value there is in this, supply's gotta go way up.

现在大家算每吉瓦成本的标准不一样，通常按五年总拥有成本来算。比方说，咱们就取70亿美元这个数吧。我觉得如果这个数字能——People give different numbers for the kind of the cost per gigawatt measured in like the five year total cost of ownership right now. But like, let's say, let's pick $70 billion as a number. I think if that number were-

我觉得要是这个数字变成200亿而不是700亿，大家都会高兴得多。而且我们每年还得要求它通过更聪明的模型，把性能再提升十倍。I think if that number were like 20 instead of 70, we should be much happier. And we should still demand to get, you know, 10x better performance out of it with smarter models every year.

为啥不好呢？我是觉得这事儿不会发生，但我们干嘛不希望成本越低越好呢？Why is it not? Like, I don't think that will happen, but why do we not want that number to be as low as possible?

我确实这么认为。对，某种意义上我支持通缩，支持通缩型技术。I do believe in that. Yeah, I'm pro-deflation in a sense, pro-deflationary technology.

每吉瓦2美元的话，我实在想象不出来。Well, $2 a gigawatt, I can't imagine.

两百亿美元。二十小时前你们的表现真不错。好吧，没错。就算成本降到二十美元，也离不开能源降价。而且，从熔化沙子到换回代币，中间那些层层叠加的毛利，肯定得大幅压缩。20 billion. 20 hours ago, you're really great. Okay, yeah, yeah. Even to get down to 20, you're getting into the realm of energy's gotta get cheaper. But certainly, like all of the gross margin stacking that happens between melting sand or the equivalent of melting sand and getting the token back, a lot of that's gotta get compressed.

我的意思是，假如材料全免费，大家也不赚钱，那靠现在的能源成本或许能勉强达标，但也仅仅是刚好够用。I mean, if you assumed like all materials were free and no one made a profit, maybe you could get there in the current energy envelope, but only just.

核心无非是智力、上下文和工具。关键要素就这几样，迟早有人能搞定。我觉得目前还没人真正摸清用户的爽点。大家觉得挺好玩，但还没到“离了它没法活”的程度。就像大家聊编程智能体，甚至聊天机器人GPT时，确实会产生依赖感。我觉得很快就会有人找到那个关键点。你刚才提到它现在还没完全跑通的原因之一。但我认为，只要再迭代几个核心想法，马上就能变成超级主流的产品。It's like intelligence, context, tools. It's a limited number of things and that someone is going. I think no one has yet figured out the like, the thing that people like. People like these, but this is not something that people are like, I cannot imagine my life without this thing.In a way that when people talk about coding agents or even ChatGPT, they do say. And someone, I think someone soon is gonna figure out what that looks like. And you talked about one of the reasons why this doesn't quite work yet. But I think this is like a small number of ideas away from a super mainstream thing.

我觉得还没有。I think we have not.

我觉得它会很像现在的多特，或者缪斯，又或者格罗克机器人。但它主打一个好用。真的就是顺手，特别靠谱。你以后几乎不用费心，只管给它派更复杂的任务，它都能稳稳搞定。I think it'll look a lot like the current, you know, Dots or Muse or Grokbot or whatever, but it'll just work.Like it'll just work, it'll be reliable. You'll like never have, you'll just like give it more complex tasks and it'll do all of those things.

我的直觉是，世界终究会走向统一。我是说，大家肯定会两头较劲，互相拉扯。但最终一定会形成共识：允许你授权AI代替你操作电脑。Like rough instinct is the world will,I mean, people are gonna fight this in both directions, but eventually converge on something like you're allowed to like tell an AI to use your computer on your behalf.

包括软件在内。Including software.

对，我也想说，这事儿没那么非黑即白。我对最终走向也没十足把握。作为消费者，要是没法让AI在自己电脑上自动干活，我会挺烦的。我就希望它像我的专属助理，听我指挥，而且我还不用分心去干别的。对。Yeah, I was gonna say, like, I don't think this is like so black and white. I don't think I have like a super high degree of confidence of how this is gonna play out. As a consumer, I feel like somewhat annoyed if I can't have an AI automate tasks on my own computer. Like I want it to feel like it's my agent doing what I ask, but then it gets like, I don't have to like go do something else. Yeah.

对，我又得先声明一下，我这儿可能有点私心，但我大方向上完全同意你。就像我买东西，就特别偏爱亚马逊（Amazon），绝对不想去那些不知名的小电商。Yeah, I mean, again, I have like a bias here, but I so directionally agree with you. Like I have such an extreme preference to buy something from Amazon versus some random e-commerce.

那些我不确定能不能退货，或者不敢保证送货快慢的商家。要是我让ChatGPT（ChatGPT）帮我买东西，能在亚马逊（Amazon）和其他平台里选的话。十次我有九次以上都会点亚马逊，哪怕多花点钱也认了。真的就是这样。这跟我平时逛亚马逊官网的习惯没关系。确实如此。现在我觉得你们可以这么说，如果合作伙伴不想发生这种情况。那我们在ChatGPT里给它们提供的平台还不够好。不过再说一次，可能我有点偏袒。我觉得亚马逊在商业布局上做得很棒。他们现在的模式更偏向智能体代理。Vendor that I don't know if I can return to or I don't know if I can like trust to deliver it quickly or something like. If I ask ChatGPT to buy me something and I can choose Amazon versus some other thing, like 10 out of 10 times I click Amazon, pay a higher price. I have just like such. Yes. And it's nothing to do with like my experience of clicking on amazon.com. Exactly.And now I think you could say we have not done a good enough job offering partners a platform inside of ChatGPT if they don't want this to happen. But, and again, maybe I'm really biased here. I think Amazon's like in a great, I think they're in a great place of commerce move, like much more agentic.

我要是能再也不用对着电脑干这种假活儿，那就太开心了。I would be so happy to never do like fake work at a computer again.

我同意。但是呢，我是最早试用那种插件的人之一。你知道的，就是那种笨拙地自动填入姓名和地址的插件。我当时就想，这活儿我再也不想干了。但现在我发现，用插件填表几乎成了上网的标准操作。I agree. But like the, I mean, I was like one of the very earliest adopters that I knew of, of those like plugins that put in your like name and address like very clumsily, you know, 'cause I was just like, this is, I cannot do this again. But I kind of feel like that using like most of the web at this point.

我们亲眼见过的最后一个例子就是应用商店，特别是iPhone的应用商店。我记得当时几乎所有创业公司，包括Y Combinator（YC）那批里的多数团队。都在说要做iPhone应用。他们认为这是全新的分发渠道，太棒了。觉得能做出以前做不到的东西。后来确实催生了一大批新公司。The last example of this that I think we really witnessed was like the app stores, the iPhone app stores specifically. And I remember every startup was like, or many startups, like most of the YC batch was like, I'm gonna build an iPhone app. This is like new distribution. This is awesome. I can do a new thing I couldn't do before. And many new startups got created.

大公司都觉得非做不可，但其实挺抵触的。他们觉得，哎呀，这不就是把跟消费者的关系交给苹果了吗？不过对我来说，这倒是件好事。And all of the big companies felt like they had to make one, but they kind of didn't want to, 'cause they're like, "Oh, I'm gonna like hand my, you know, relationship with my consumer over to Apple." And it was great for me.

我觉得我们该关注两件事。一是科学发现，以及人类做不到的事。二是合作起来是否愉快。我觉得现在没有很好的标准来衡量“好不好用”。甚至顶尖科学家的水平，好像也没法完全靠测试来定。但我们尽量不依赖基准测试。我一直觉得，这种现象早就该过去了，可大家还在反复做。I mean, maybe there are like two things we should really care about models being good at. One is like, you know, scientific discovery and doing things that people cannot do. And the other is like being enjoyable to work with. And I don't think we have great benchmarks for being enjoyable to work with.I'm actually not sure we have great benchmarks for like the smartest scientific scientist either. But we endeavor not to benchmark. And I kind of, I keep thinking that benchmarking is like over as a phenomenon and people keep doing it.

我相信是的。说实话，回顾历史，我们未必一直做得最好。但我认为现在的我们确实更强了。I believe we are better at that.It feels like- I don't think we've always been better at that in our history. I believe we are better than that right now.

确实进步很大。我觉得最新的模型，比过去很长一段时间里的都要好得多。It's gotten, I think the latest models are much better than we've had in a long time.

几年前埃米特·希尔（Emmett Shear）跟我说，他想要一位资深同事。这位同事既要深思熟虑、敢于挑战，又要保持尊重。至今我没听过比这更精炼的总结了。Emmett Shear said to me years ago that what he wanted was a really thoughtful and challenging, but still respectful senior coworker. And I have still not heard a better like brief encapsulation than that.

我觉得格雷格·布克曼是独一无二的，而且是那种最好的独一无二。I think Greg Brockman is like one of a kind in the best sense.

我觉得这是一种结合。极强的专注力，惊人的生产力，极度的投入，还有对核心使命的死磕。真的非常酷。But I think it's like a combination of like, yeah, intensity, like monster productivity, extreme dedication, and like focus on 60th mission. It's very cool.

说实话，我们确实讨论过——You know, we did talk--

我们差点就做了个蠢决定，说下一代模型完全不按词元定价。你就给个提示词，我们直接回报价。就像亚马逊云服务（AWS）那样，你只管用，我们按月结账。但客户反应特别差。他们说：总得让我知道钱花在哪了吧。我问了几个人：如果输出一样，你是宁愿多花钱，但按固定的词元数量计价？哪怕我们把词元数量虚高一点。我肯定选虚高词元然后多付钱。因为我知道背后有实实在在的成本。所以你说得对，大家不只想让我们随便报个数。长远看，他们怕吃亏。就像大家宁愿按处理器时长付费，也不愿只管把事做完。哪怕前者更贵。We almost made a very, what would've been bad call to say we're not gonna price our next model in tokens at all.And you're just gonna like give it a prompt and we're gonna give you a price back and we're gonna do it. And it's gonna be like AWS, you just do something and we send you a bill. But the customer response on that is so negative. They're like, "I need to know what I'm paying for somehow." And I asked a few people like, "Would you rather pay more for the same output, but have it priced in some number of consistent tokens or something? Like even if we like inflated the tokens." And I'd much rather inflate, you inflate the tokens and pay more, 'cause I know there's like a real thing there.So I think you are right that people don't just want us to like give them a price because they feel like over the long term they're gonna get screwed somehow.In the same way that maybe people would like rather pay for CPU hours rather than just like getting their thing done. And even if it was less expensive.

指那种最小特征尺寸的。这完全是硬凑的词。我希望别这样。说实话，我很反感拿纳米这个词，去形容跟纳米毫无关系的东西。Of like the smallest feature.That's like made up. I hope not.I mean, it offends me that we like use the word nanometers for something that has nothing to do with nanometers.

我宁愿不造这种假单位。词元本身就算一种假单位。它在某种意义上是真实的，但作为计量单位并不好。I would rather not make up a fake unit.Tokens are a kind of fake unit. I mean, they're real in some sense, but like they're not a good unit.

对。现在你得纠结，是用小模型每百万词元只要一毛钱，还是大模型每个词元十美元。还得猜思维链能有多长。这纯粹是增加认知负担，我觉得根本不值当。Yeah, I mean, right now you have to think about like whether you should want like the 10 cent per million tokens out of a small model or $10 per token, tokens out of a big model. And then whether like how long the chain of thought is likely to be. And like that's just like cognitive overhead that I don't think actually adds any value.

在我们目前的计费方式里，大部分都包含在内。In what we charge, most people do.

顺便说一句，健力士（Guinness）啤酒其实没我想的那么难喝。肯定不如印度淡色艾尔（IPA），但也没那么糟。挺好喝的。IPA确实很棒，但这款也挺不错。By the way, the Guinness was not as bad as I thought. It definitely not as good as an IPA, but not as bad. It's good. Yeah, I mean, an IPA is great, but it's pretty good.