In this podcast, Keras creator François Chollet argues that the idea of an 'intelligence explosion' (AI rapidly self-improving to superintelligence) is flawed. He says intelligence isn't a standalone property of the brain but emerges from brain, body, and environment. Science itself is a counterexample: despite exponentially more resources, major discoveries haven't exploded. He also explains that deep learning can only interpolate near its training data, not generalize like humans. He notes today's self-driving cars are mostly hand-coded rules, not pure AI. The biggest AI risk, he says, isn't superintelligence but recommendation algorithms (like social media feeds) that can manipulate behavior at scale.
François Chollet (creator of Keras and Google AI researcher) discussed deep learning, the Keras library, and the future of AI on the Lex Fridman podcast. The core argument challenges the "intelligence explosion" theory—the idea that AI could recursively self-improve, leading to exponential growth in
François Chollet (creator of Keras, Google AI researcher) systematically challenges the theory of "intelligence explosion" on the Lex Fridman podcast, and discusses the capability boundaries of deep learning, the design philosophy of Keras, the social risks of AI, and a new framework for measuring intelligence.
The most substantive judgment in the episode: François Chollet argues that the reasoning behind "intelligence explosion" is fundamentally flawed — intelligence is not an isolated property of the brain, but an emergent characteristic of the brain-body-environment system; any system undergoing recursive self-improvement inevitably encounters exponential friction, and science itself serves as a living counterexample.
Chollet argues that the "intelligence explosion" theory (recursive self-improvement in AI leading to exponential growth in intelligence) is logically untenable.
Chain of reasoning:
1. Flawed definition of intelligence: The theory implicitly treats intelligence as "an isolated property of the brain, measurable independently like the height of a building." Chollet counters: "Intelligence emerges from the interaction between a brain, a body, like embodied intelligence, and an environment. If you're missing one of these pieces, then you cannot really define intelligence anymore."
2. Law of bottlenecks: In any recursive self-improving system, improving one component immediately creates a new bottleneck elsewhere. Chollet uses a physical analogy: "If you build a device that measures its own acceleration and outputs greater acceleration, then drop it in the air, it will not reach infinite speed — because the surrounding air creates friction, limiting it to a terminal velocity."
3. Science as a real-world case: Science itself is a recursive self-improving problem-solving system — scientific progress drives technology, and technology (computers, the internet) accelerates science. Yet scientific progress has not exploded. Chollet cites Michael Nielsen's research: measuring the "importance time density" of scientific discoveries over the past 100–150 years reveals a flat curve across all disciplines (physics, biology, medicine). Meanwhile, resources consumed by science (number of researchers, computing power) have grown exponentially. Conclusion: Exponential resource consumption yields linear progress, because each step forward makes the next step exponentially harder.
4. Mapping to deep learning: Chollet believes the same holds true in deep learning — the number of papers grows exponentially, but the "importance per paper" declines, with overall progress remaining roughly linear.
Extrapolation and falsification condition: If a recursive self-improving system emerges in the future whose output (rather than resource consumption) grows exponentially, Chollet's judgment would be falsified. But he considers this impossible because "exponential progress triggers exponential friction."
Reader's note: Chollet explicitly states this is "intuition, not a mathematical proof," but he believes the "intelligence explosion" is more of a belief system than a scientific argument. Questioning it is akin to challenging a religious narrative, and thus invites strong backlash.
Chollet argues that deep learning is essentially a "pointwise continuous geometric deformation," which prevents it from achieving human-like extreme generalization.
Mechanism Breakdown:
Data Chain and Comparison:
| Feature | Deep Learning | Abstract Rules / Symbolic Systems |
|---|---|---|
| Learning Method | Pointwise geometric mapping | Symbolic manipulation / programs |
| Generalization Ability | Local interpolation | Extreme generalization |
| Data Requirement | Dense sampling (potentially trillions of samples) | Few examples |
| Applicable Domain | Perception problems | Reasoning / planning |
Success Case: Chollet notes that the most successful AI systems today (e.g., autonomous vehicles) are already hybrid systems — deep learning serves as the perception module, while symbolic systems (planning algorithms, 3D models) handle reasoning. "An autonomous vehicle is not one big end-to-end neural network; that simply wouldn't work. It is mostly symbolic, it is hand-written software."
Extrapolation: Chollet believes the future direction of AI lies in the combination of deep learning and program synthesis. Program synthesis is currently in a "pre-backpropagation winter" — we have yet to find its "engine." However, he is convinced that "discrete search over program synthesis and rule-based models will become the cornerstone of AI research in the next century."
Chollet articulated Keras’ design principle—the API should reflect the mental model of domain experts, not implementation details.
Historical Context:
Design Philosophy: Chollet believes the key to API design is "minimizing the cognitive load for domain experts to map their mental models to the API." "The API should not be self-referential or refer to implementation details; it should only reference domain-specific concepts that people already understand."
Breakthrough in TensorFlow 2.0: It achieved a complete spectrum from low-level (subclassing models, custom training loops) to high-level (classic Keras workflow), allowing users to trade off between flexibility and ease of use based on their needs.
Future Direction: Chollet is interested in higher-level APIs—"Automated Machine Learning (AutoML), a model that can automatically look at data and optimize objectives, much like putting a child who is good at LEGO together with a box of LEGO."
Chollet argues that the most realistic threat posed by current AI is not superintelligence, but large-scale behavioral manipulation through recommendation algorithms.
Mechanism Breakdown:
Chollet's Solution: Users should be able to control the algorithm's objective function. "I want all algorithms to present users with configuration settings, allowing users to decide how the algorithm influences their lives—for example, choosing 'maximize learning' over 'maximize engagement.'" He views this as a design interface issue, not a technical one.
Critique of the Industry's Current State: Chollet points out that many companies and individuals are overhyping AI capabilities (e.g., promising Level 5 autonomous driving by 2021–2022), which amounts to "taking on credit card debt on trust." When promises fail to materialize, it could trigger a collapse of trust akin to an AI winter. However, he believes a full-scale AI winter is unlikely, as deep learning is generating real value.
Chollet proposes an operational definition: intelligence is the efficiency with which experience is transformed into generalizable programs.
Definition Expanded: Given the same innate prior knowledge and the same amount of experience, the agent that can learn from that experience to produce a program that performs well across a larger space of experiences is the more intelligent one.
Key Distinctions:
Critique of Existing Benchmarks: Chollet argues that skill-based tests such as Go and Dota do not measure intelligence because they fail to control for prior knowledge and the amount of experience. "If you remove the constraints—a lifetime, physical limitations—then achieving any arbitrary level of skill is neither interesting nor indicative of how much intelligence you have attained."
Vision for a New Benchmark: Chollet is developing a benchmark whose core ideas are:
1. Explicitly list the innate prior knowledge humans are born with (e.g., object permanence, time, agency, goal-directed behavior)
2. Create new tasks that rely solely on these priors
3. Control the amount of experience and compare the performance of humans and AI systems
4. Key design feature: Scores should not improve regardless of how long one practices—because the tasks themselves are entirely novel
Insight on Human Priors: Chollet notes that DNA is an extremely low-bandwidth medium, and encoding any knowledge requires millions of years of evolutionary pressure. Consequently, human innate knowledge is very limited, and most of it is shared with close relatives (great apes)—"What makes us special is not our innate knowledge, but something else."
This section is an interview/discussion of views and does not involve specific investment targets or position actions.
1. Chollet argues that "intelligence explosion" is a matter of faith, not science: "For many, AI is not just a subfield of computer science but more like a belief system—the conviction that the world is heading toward a singularity. Questioning it is akin to saying God does not exist." Support: The implicit definition of intelligence underlying this theory is flawed, and the real-world manifestation of science as a recursive self-improving system is linear progress, not exponential explosion.
2. Exponential progress triggers exponential friction: Chollet uses a physics analogy to illustrate—improving one part of a system immediately makes another part the new bottleneck. The resources consumed by science grow exponentially, but output (the time density of significance) follows a flat curve across all disciplines.
3. The essence of deep learning is "pointwise geometric mapping": It can only interpolate the space near the training data and requires dense sampling. Abstract rules (e.g., sorting algorithms) can handle arbitrary inputs because they are symbolic. The future lies in combining the two.
4. The most successful AI systems today are already hybrid systems: Autonomous vehicles "are primarily symbolic, consisting of hand-written software," with deep learning serving only as a perception module. End-to-end neural networks are infeasible for complex real-world problems.
5. Program synthesis is in a "winter before the invention of backpropagation": We have yet to find its "engine," but it will become a cornerstone of AI research in the next century.
6. DNA is an extremely low-bandwidth medium: Human innate knowledge is very limited, and most of it is shared with great apes. This makes it possible to construct fair intelligence benchmarks—explicitly listing human priors and then comparing AI and human performance under the same conditions.
7. The greatest risk of recommendation algorithms is not malice but objective function design: With the goal of "maximizing engagement," even without malicious intent, the algorithm naturally biases toward fake news and extreme content. The solution is to let users control the algorithm's objective function.
8. Overhyping AI capabilities is "taking on credit card debt for trust": Promising L5 autonomous driving by 2021–2022, which remains unfulfilled, is eroding industry credibility. A full-scale AI winter is unlikely (deep learning creates real value), but localized winters and a collapse of trust are inevitable.