AI 认知漏斗

Dusting off this tweet from April because this gap in shared understanding of LLM capability is widening. It's now less "two groups of people speaking past each other" and more a sharp funnel.

• Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
• Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
• Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
• And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.

The funnel is driven by a combination of factors:

1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.

So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.

把 4 月的这条推文翻出来，是因为人们对大模型能力的共同认知，这道鸿沟正在扩大。现在它已经不太像“两拨人各说各话”，而更像一个尖锐的漏斗。

• 粗略估算，大约 60 亿人（约占人口的 75%）几乎完全没接触过大模型。
• 大约 10–20 亿人（约 20%）是免费版 ChatGPT 类产品的轻度、偶尔用户。这群人看到的是有点傻的聊天机器人，把它们当成更好用一点的谷歌搜索、写作助手之类。也许会有个智能体试着帮你订机票。我有一些不做技术的朋友说，他们其实用不太上它（我觉得这挺合理）。甚至很多数学和代码以外的专业人士也在这一层。比如高管和许多其他岗位，大量时间在和人沟通，所以虽然他们在理智上明白正在发生什么，但那仍是二手的、有点抽象的。
• 接下来是在数学和代码领域专业使用前沿级大模型的人。大约 2000 万人（0.2%）亲眼看到：过去要花他们几周、几个月的大型复杂项目，现在用一个提示词就能交给智能体完成。开发应用、复刻应用、翻译应用、从二进制文件反编译应用……这一切都发生得非常快、非常近——不到一年前，我还在手工写代码，把记住的代码命令一个字符一个字符地敲进编辑器，偶尔按一下 Tab 自动补全一小段。
• 最后是大约 5000 人（约 0.00006%），他们能在内部使用前沿级系统。外部世界已经看到了预告片：成群的、成千上万个智能体在软件巨型项目上协作数周——挖出零日漏洞、以机器速度进行网络攻击与防御、发现新的科学、推进数学的前沿。这些事情原本需要业内顶尖专业人士花上好几年。与此同时，人类的审查和理解开始跟不上了。比如，人们至今仍在对几个月前的 OpenAI-HF 事件做“考古”。数学家们可能要花很长一段时间，钻研那 722 篇前沿数学手稿。

这个漏斗由几个因素共同驱动：

1. 影响力随野心、问题规模和时间跨度而放大。一个用一段话就能回答的问题，几乎不会给系统施加压力。你需要一池你真正在乎的、大而难的问题。这一点驱动了漏斗中“消费者 / 专业人士”这一维。
2. 系统的锯齿性（我另外写过很多）。能力在数字化、可验证、经济价值高的领域达到顶峰。这是因为大模型的能力来自：在一个按收入潜力精心挑选的环境组合上，用可验证奖励进行强化学习。这一点主要驱动了漏斗中的“领域”（例如数学和代码）这一维。
3. 访问权限。免费版、付费版、内部。

这就是此刻的怪异之处。公众大多没有接触过这些系统；接触过的，看到的是一个有点傻的聊天机器人；大多数专业人士仍然只看到温和的提升；而一小部分专业人士正在经历曲线变得垂直的眩晕。而这一切，都在同一时刻发生。