← Back to list
Lex Fridman PodcastPodcast26 Nov 2022Source: lexfridman.comHost: Lex Fridman

#341 – Guido van Rossum: Python and the Future of Programming

In plain words

This interview covers Python creator Guido van Rossum's views on the language's present and future. The big takeaway: Python 3.11's 10-60% speedup comes not from a JIT compiler but from an 'adaptive specializing interpreter' that watches variable types at runtime and creates shortcut instructions for common patterns. He's bullish on VS Code (his daily driver) and GitHub Copilot (uses it daily to save typing), but lukewarm on PyCharm, calling it 'like driving an 18-wheeler'—powerful but unwieldy.

AI SummaryAI-generated · may contain errors · verify against the original

This report is based on Episode 341 of the Lex Fridman Podcast, featuring a discussion by Python creator Guido van Rossum on the Python language and the future of programming. The core argument is that Python, as a concise and easy-to-use language, relies heavily on its design philosophy (such as th

~13 min full read · 8 sections
Deep Analysis

At a Glance

Guido van Rossum, the creator of the Python programming language and currently a senior developer at Microsoft, is the focus of this issue. The core discussion revolves around Python's design philosophy, performance evolution, and future direction. The most significant insight in the entire episode is that the 10-60% performance improvement in Python 3.11 was achieved not through a JIT compiler, but via an "adaptive specialization interpreter"—which dynamically identifies variable type patterns at runtime and generates specialized instructions. In essence, it replaces generic paths with statistical prediction, marking a key breakthrough in dynamic language performance optimization.


1. Python's Design Philosophy: Readability Above All

Guido van Rossum believes that the core contradiction of programming languages lies in "the same piece of code needing to be readable for both computers and human readers." Python defines code blocks through mandatory indentation (rather than curly braces)—a design choice that was radical 30 years ago and remains Python's most distinctive feature to this day.

  • Historical Context: While learning programming in the 1980s, Guido observed that most languages (C, Java, JavaScript) use curly braces to denote block structures, but beginners often fail to compile due to missing semicolons or parentheses. Python uses indentation to eliminate such syntactic noise, allowing beginners to "worry about one less thing."
  • Mechanism Breakdown: Indentation is not merely a stylistic suggestion but a part of the language's syntax. Guido admits that "if designing from scratch today, I would still choose indentation—it frees up curly braces for other uses (such as dictionaries) and makes code structure immediately clear." However, he also notes that Python has become "an outlier," as virtually all mainstream languages use curly braces.
  • Data Support: The PEP8 style guide recommends 4-space indentation, a compromise based on 1980s screen width and readability experience. Google's internal Python code uses 2 spaces, which Guido believes "makes it harder for those unaccustomed to it to grasp the code structure at a glance."

Unique Insight: Guido compares programming languages to "recipes written for both computers and humans"—computers require precise instructions, while humans need readable structure. Python's design consistently prioritizes the latter, which is the key reason for its popularity among beginners.


2. CPython 3.11 Performance Breakthrough: Adaptive Specializing Interpreter

Guido van Rossum points out that the 10-60% performance improvement in version 3.11 comes from the "adaptive specializing interpreter," not a JIT compiler. The core idea is to observe variable type usage patterns at runtime. When a line of code repeatedly handles the same type (e.g., integer addition), specialized instructions are generated to bypass the generic lookup path.

  • Mechanism Breakdown:
  • Python is a dynamically typed language, so `a + b` does not know the types of `a` and `b` at compile time. The traditional implementation requires: retrieving the type from the object → retrieving the addition function pointer from the type → calling the function → checking parameter types → executing the operation.
  • In 3.11, the approach is: each time an addition is executed, the actual types of the operands are recorded. If the same type (e.g., integer) appears consecutively multiple times, a "specialized integer addition" instruction is generated. This instruction still checks the type (in case a string appears later), but the check path is extremely short, and statistically, 99% of checks pass.
  • Key Design: If the prediction fails (e.g., a float suddenly appears), the interpreter falls back to the generic path. Guido admits that "theoretically, one could construct code that runs 5 times slower than 3.10, but that is highly unrealistic."
  • Data Chain: CPython is the reference implementation of Python, written in C. All optimizations in 3.11 are concentrated at the interpreter level, with minimal changes to the compiler (the part that compiles Python source code into bytecode).
  • Falsification Condition: If a line of addition code in a given segment frequently switches types (e.g., alternating between integers, floats, and strings), the adaptive mechanism will frequently fall back, potentially leading to no performance gain or even a decline. However, Guido believes "this pattern is extremely rare in real-world code."

Unique Insight: Guido uses the analogy of "weather prediction"—assuming tomorrow's weather will be the same as today's, this simple heuristic is already much better than random guessing. The optimization in Python 3.11 essentially assumes "the behavior of the next line of code will be the same as the previous line," which is the core philosophy of dynamic language performance optimization.


3. The Present and Future of Type Hints: A Laboratory, Not a Language Core

Guido van Rossum believes that type hints (PEP 484) represent "the most active experimental field" in the Python ecosystem, but they will not become part of the language core in the short term. Currently, 20-30% of Python 3 codebases use type hints, primarily in continuous integration workflows at large companies.

  • Historical Context: The syntax for type hints (e.g., `def foo(x: int) -> str:`) was reserved in Python 3.0 (function annotations), but its purpose was not clarified until PEP 484 (2014). Guido and Finnish developer Jukka Lehtosalo reached a compromise at the 2013 Python conference: use the existing annotation syntax and avoid disruptive changes such as introducing angle brackets (like C++ generics).
  • Mechanism Breakdown:
  • Type hints are "optional and introspectable at runtime"—annotation information can be accessed via `__annotations__` at runtime, but the interpreter does not use it for any optimization or checking.
  • Static type checkers (e.g., MyPy, Pyre, PyType, PyRight) are independent tools that run during development. Guido emphasizes that "if the interpreter started enforcing type annotations, many existing Python programs would break, because annotations may contain lies."
  • Competitive Landscape:
  • MyPy (the original checker, written in Python) remains the most popular, but Google (PyType), Facebook (Pyre, written in OCaml), and Microsoft (PyRight) have all developed their own checkers.
  • Guido believes that "having multiple checkers is a good thing—they evolve at a pace of monthly or bimonthly releases, much faster than the Python language itself (once a year), and serve as a laboratory for syntactic innovation."
  • Uncertainty: Guido explicitly states that "no one is actively pushing to integrate a type checker into the language right now," but "the situation could be different in 5-10 years."

Unique Insight: Guido likens the current state of type hints to "competition among JavaScript engines"—multiple implementations coexist and drive innovation, while the language itself remains neutral. This contrasts with TypeScript's "preprocessor model."


4. GIL’s Dilemma and Possible Future: Sub-Interpreters More Realistic Than No-GIL

Guido van Rossum believes that the cost of removing the Global Interpreter Lock (GIL) may outweigh the benefits, and a more realistic path is the "multi-sub-interpreter" approach, expected to be introduced in Python 3.12 (approximately one year from now).

  • Historical Context: The GIL was introduced in the early 1990s as a "shortcut" to quickly support multithreading—at a time when multi-core CPUs were not yet widespread, the GIL allowed multithreading to appear functional on single-core systems. As multi-core processors became mainstream, the GIL prevented Python multithreading from leveraging multiple cores (all threads effectively run on a single core).
  • Mechanism Breakdown:
  • The essence of the GIL is "using a single global lock to protect the entire interpreter state," preventing data races caused by multiple threads modifying objects simultaneously. The cost is that only one thread can execute Python bytecode at any given moment.
  • Sub-interpreter approach: Each sub-interpreter runs a completely independent Python program with its own GIL, thus enabling true parallelism. However, sub-interpreters cannot share objects, and communication costs are higher.
  • No-GIL approach: Facebook developers have already created a "no-GIL" branch, which keeps single-threaded performance degradation manageable through extensive optimization. But Guido argues that "maintaining this complexity is not worth it—the GIL is a reasonable compromise."
  • Data Support: Guido cites his own blog post on semaphores, noting that "the bug density in concurrent programming is far higher than in sequential code—the human brain is not good at tracking multiple execution flows simultaneously." The GIL effectively protects developers from the worst concurrency issues.
  • Falsification Conditions: If the maintenance cost of the no-GIL branch can be accepted by the community and single-threaded performance loss is kept within 5%, Guido does not rule out making it a core feature of Python 4.0. However, he emphasizes that "Python 4.0 will not arrive soon, and even if it does, it will be fully syntax-compatible with 3.x."

Unique Insight: Guido compares the GIL to the "Goldilocks point"—neither no threads (too primitive) nor fully free threading (too dangerous). This judgment stands in stark contrast to many community views that pursue extreme parallelism.


5. Python’s Dominance in AI/Data Science: Coincidence and Inevitability

Guido van Rossum believes that Python’s rise as the dominant language for machine learning and data science is the result of path dependence akin to the "right-hand driving rule" combined with an "open community culture," rather than any inherent design advantage of the language itself.

  • Historical Context:
  • In the late 1990s, Paul Dubois of Lawrence Livermore National Laboratory proposed the concept of "computational steering": scientists needed a high-level language to orchestrate Fortran/C++ numerical libraries. Python became a candidate due to its extensibility.
  • The Hubble Space Telescope team made extensive use of Python in the late 1990s. The emergence of array manipulation libraries such as NumPy and SciPy allowed Python to take root in scientific computing.
  • Frameworks like TensorFlow and PyTorch chose Python as their user interface because "scientists were already familiar with Python"—a classic case of path dependence.
  • Competitive Comparison:
  • MATLAB’s failure was not due to technology but to being "closed-source, expensive, and lacking GitHub’s open-source culture"—it could not achieve "viral spread."
  • Perl dominated bioinformatics in the early 2000s (thanks to its regex strengths) but failed to build an array computation infrastructure.
  • Community Culture: Guido emphasizes that the Python Software Foundation (PSF) allocates its funds "almost entirely to community building, not development"—this "egalitarian" culture allows individual developers and small companies to contribute, creating a positive feedback loop.

Unique Insight: Guido uses the "right-hand driving rule" analogy—Python’s position in AI is not the optimal solution but rather the result of "everyone happening to choose the same side." This judgment challenges the popular narrative that "Python won because of its technical advantages."


Mentioned Positions

Position Guest Attitude Key Data
CPython Core Focus Version 3.11 performance improvement of 10-60%; 30-year history, implemented in C
MyPy Bullish Original Python static type checker; co-developed with PEP 484
PyCharm Neutral (with reservations) Most feature-rich but "feels like driving an 18-wheeler"; extension development is difficult
VS Code Bullish Viewed as the "spiritual successor" to Emacs; Guido uses it daily
GitHub Copilot Active User "Used daily, saves a lot of typing"; but "you need to understand the code to use it well"
NumPy/SciPy Positive Mention Key infrastructure for Python's foothold in scientific computing
TensorFlow/PyTorch Positive Mention Chose Python as the UI because scientists were already familiar with Python

Judgments Worth Remembering

1. Guido van Rossum believes the "adaptive specialization interpreter" is the core philosophy for optimizing dynamic language performance: It assumes tomorrow's weather will be the same as today's—using statistical predictions to replace general-purpose paths, betting that variable types remain unchanged in most cases. The 10-60% speed improvement in version 3.11 comes from this approach, not JIT.

2. Guido considers the GIL a "Goldilocks point": It is neither without threads (too primitive) nor fully free-threaded (too dangerous). The maintenance cost and single-thread performance loss of removing the GIL may outweigh the benefits, making sub-interpreters a more realistic path.

3. Guido notes that type hints will not become a core language feature in the short term: Currently, 20-30% of Python 3 codebases use type hints, but the interpreter does not enforce checks. The coexistence of multiple static checkers (MyPy, Pyre, PyType, PyRight) serves as a "laboratory for syntax innovation."

4. Guido argues that Python's dominance in AI is a "right-hand rule" path dependency: It is not technically optimal but rather "everyone happened to choose the same side." MATLAB failed due to being closed-source and expensive; Perl failed due to lacking array operation infrastructure.

5. Guido compares programming languages to "recipes written for both computers and humans": Computers need precise instructions, while humans need readable structure. Python's mandatory indentation prioritizes the latter, which is a key reason for its popularity among beginners.

6. Guido suggests that "the benefit of the BDFL model is directional stability, while the downside is excessive personal pressure": He admits he "should have given up the BDFL role earlier," but the successor steering committee has successfully maintained an evolutionary pace that is "stable without stagnation."

7. Guido believes "the best way to learn programming is to find a specific problem you want to solve": Even impractical problems (like writing a Reddit bot) are more effective than abstractly learning syntax. He references the criticism of "Learn Python in 10 Years" but argues that "one hour can make you fall in love with programming—the key is to fall in love first, then go deeper."

8. Guido views VS Code as the "spiritual successor" to Emacs: Both adopt a "core engine + extensible package" architecture, and VS Code's package ecosystem is a modern version of the Emacs Lisp tradition. This is why he switched from Emacs to VS Code after joining Microsoft.