Zhipu Tang Jie: Scaling Beyond Parameters

Welcome back to Learn English with Podcasts! Sarah, if a company told you their new AI has 1 trillion parameters, would you say wow, it must be the best?欢迎回到 Learn English with Podcasts！Sarah，如果一家公司告诉你他们的新 AI 有 1 万亿参数，你会说哇，它一定是最好的吗？

That is exactly what Zhipu founder Tang Jie wants to challenge. He just posted his Thoughts About Scaling Law, and his first line is simple - Scaling, but not just parameters.这正是智谱创始人唐杰想挑战的观念。他刚发布了关于 Scaling Law 的思考，开头就很简单——要 Scaling，但不只是参数。

We are scaling an incomplete picture. Tang Jie says reporting only the parameter number tells you almost nothing today. You also need to ask: how much data, where you spend your compute, and how the model will run every day.我们看到的是不完整的图景。唐杰说，今天只报参数量几乎说明不了什么。你还得问：用了多少数据、把算力花在哪，以及模型每天是怎么运行的。

It started in 2020. Researchers led by Kaplan tested many model sizes and found a rule. When you have more compute, you should grow parameters faster than data. About 2.7 to 1.从 2020 年开始。Kaplan 带领的研究者测试了很多模型尺寸，发现一条规律：算力更多时，参数应该比数据涨得更快，大约是 2.7 比 1。

Completely. That rule set the race. GPT-3 came with 175B parameters. Then Gopher, then MT-NLG with 530B. Everyone started chasing 1 trillion.完全照做。这条规则定下了竞赛方向。GPT-3 带着 175B 参数登场，然后是 Gopher，再到 530B 的 MT-NLG。大家都开始追逐 1 万亿。

And then in 2022, DeepMind did the test again. But bigger. They trained over 400 models, from 70M to 16B parameters, on 5B to 500B tokens.然后在 2022 年，DeepMind 又把这个实验重做了一遍。但规模更大。他们训练了超过 400 个模型，参数从 70M 到 16B，数据从 50 亿到 5000 亿 Token。

And they found we were wasting resources. Models had too many parameters and not enough data. So they built Chinchilla.然后他们发现我们一直在浪费资源。模型参数塞得太多，吃进去的数据却不够。所以他们造了 Chinchilla。

Exactly. Chinchilla is only 70B parameters, 4 times smaller than Gopher at 280B. But it was trained on 1.4T tokens, 4 times more data. With the same training compute, the smaller, better-fed animal won. It beat Gopher, GPT-3, Jurassic-1, and MT-NLG.没错。Chinchilla 只有 70B 参数，比 280B 的 Gopher 小 4 倍。但它用了 1.4T Token 训练，多 4 倍数据。用同样的训练算力，更小但吃得更饱的选手赢了。它击败了 Gopher、GPT-3、Jurassic-1 和 MT-NLG。

Yes. Chinchilla Scaling Law. But Tang Jie says even that was not the final answer.对，这就是 Chinchilla Scaling Law。但唐杰说，那也不是最终答案。

Because training is only done once, but running the model happens billions of times a day. A 1 trillion parameter model needs huge compute every single time you ask it something. Chinchilla did not count that daily bill.因为训练只做一次，但运行模型每天要发生数十亿次。1 万亿参数的模型每次你提问都要消耗巨大算力。Chinchilla 没算这笔日常账单。

Perfect metaphor. Some studies looked at the full life. If a model will handle about 1B requests, a smaller model trained longer can be cheaper overall. They pushed training to 10,000 tokens per parameter and quality still kept improving.这个比喻完美。一些研究看了全生命周期。如果一个模型要处理约 10 亿次请求，一个更小但训练更久的模型总成本可能更低。他们把训练强度一路推到每个参数 1 万个 Token，质量还在提升。

Sure. Llama 2 7B was trained on about 290 tokens per parameter. Gemma 2 9B reached about 889 tokens per parameter. Way beyond the old 20.可以。Llama 2 7B 每个参数约对应 290 个训练 Token，Gemma 2 9B 更达到约 889 个。远超当年的 20。

Exactly. And for today's MoE models, it moves again. Tang Jie splits two ideas: total params and active params.没错。而对于今天的 MoE 模型，它又变了。唐杰把两个概念分开：总参数和激活参数。

He uses a similar picture. Total params is the warehouse - how much knowledge you can store. Active params and effective depth is how much power you use in one go, how far you can think in one chain.他用了类似的比喻。总参数是仓库——能装多少知识。激活参数和有效深度是单次能调动多少能力，能把一条推理链走多远。

And bug hunting shows the difference. Remembering many CVEs means you saw many cases. But to find a new bug, you must follow the code for 20 steps - from strange code to how it breaks, to how to test it - without losing the thread.而抓漏洞最能看出区别。记住很多 CVE 意味着你见过很多案例。但要发现一个新漏洞，你得把代码跟上 20 步——从异常代码到触发条件再到验证方法——全程不能掉线。

So Tang Jie says you cannot judge that long-chain skill by total size alone. And that brings us to GLM-5.3.所以唐杰说，你不能只用总规模去判断这种长链能力。这就说到了 GLM-5.3。

Yes. For GLM-5.3, Zhipu kept the base, the architecture, total params and active params all the same as GLM-5.2. No bigger warehouse. For the past month, they put new compute into long-horizon environments and reinforcement learning.是的。做 GLM-5.3 时，智谱保留了 GLM-5.2 的基座和架构，总参数和激活参数都没变。没有把仓库变大。过去一个月，他们把新增算力都投进了长程任务环境和强化学习。

Great way to say it. The gains came from post-training. Official numbers: Terminal-Bench 3.0 went from 4.6 to 28.3, DeepSWE from 46.2 to 66.9, and Agents' Last Exam from 23.8 to 28.5.说得太好了。提升来自后训练。官方数字：Terminal-Bench 3.0 从 4.6 升到 28.3，DeepSWE 从 46.2 涨到 66.9，Agents' Last Exam 也从 23.8 升到 28.5。

It is. Now, these are official tests, outside teams will need to check again when weights are open. But for Tang Jie, it proves a point - scaling has many knobs.确实很大。当然这些目前主要是官方测试，外部复现还得等权重开放。但对唐杰来说，这已经证明一点——Scaling 有很多旋钮。

Exactly. GLM-5.3 turned the post-training knob because that knob had the most room left. Next time, Zhipu might turn pre-training or mid-training or compute per forward pass.没错。GLM-5.3 拧了后训练这个旋钮，因为当时这里剩余空间最大。下一次，智谱可能再去拧预训练、中训练或单次前向算力。

Beautiful summary. The game is moving from who has the biggest library to who can finish a real job - work for hours or days like a digital employee, without dropping the ball.总结得漂亮。竞赛正从谁的藏书最大，转向谁能把一份真实工作做完——像数字员工那样连续工作几小时、几天，不掉线、不跑偏。

And Tang Jie closes with a calm line: 1 trillion params has not disappeared. But from now on, it may not be the only number worth putting in the headline.唐杰最后也平静地收了尾：万亿参数没有消失。只是从现在开始，它可能不再是唯一值得写进标题的那个数字。

And that is the quiet joke of this story. For three years we counted params like height. Turns out AI is not a basketball team. The tallest player does not always win the marathon.这也是这个故事里安静的幽默。三年里我们像量身高一样数参数。结果发现 AI 不是篮球队，最高的不一定赢得了马拉松。

Thanks for listening to Learn English with Podcasts. Try asking your next AI not how big it is, but how many steps it can stay with you.感谢收听 Learn English with Podcasts。下次问你的 AI 时，别问它多大，问问它能陪你走多少步。