#447 – Cursor Team: Future of Programming with AI

This Issue at a Glance

The Cursor team (Aman Sanger, Arvid Lunnemark, Michael Truell, Sualeh Asif) delved into the future of AI-assisted programming on the Lex Fridman podcast. The core thesis is: AI will fundamentally change how programming is done, but human programmers will remain in the driver's seat for the long term. The key lies in building a "human-machine hybrid engineer"—by leveraging custom models, intelligent UX, and ultra-low latency, programmers iterate at the speed of judgment rather than being replaced by AI. Michael Truell's assessment: "The best product 3-4 years from now will be far more useful than the best product today—the ceiling in this field is extremely high, and stopping innovation means losing."

From Copilot to Cursor: Why Forking VS Code Was Necessary

Michael Truell argues that the capability leap of GPT-4 turned "AI will change all of programming" from theory into reality, demanding an entirely new programming environment, not just a plugin.

• Historical Context: The team were originally Vim users, switching to VS Code after Copilot's 2021 release. Copilot was "the first killer app for LLMs"—even when it guessed wrong, one keystroke to correct it made the low-friction experience captivating.
• Key Turning Point: Gaining early access to GPT-4 in late 2022. "The capability improvement felt enormous... Before that, we were building point solutions like Jupyter tools for finance professionals and static analysis. GPT-4 convinced us that all of programming would flow through these models."
• Why Fork Instead of Plugin: Plugins have extremely limited control over the editor. "We didn't want to be locked in by the constraints of an existing environment; we wanted to build the most useful thing possible."
• Competitive Logic: Aman Sanger notes that in the AI coding space, "every year, every model capability leap unlocks a new wave of features. Being a few months or even a year ahead creates a huge gap in product usefulness. The Cursor version one year from now must make today's Cursor look outdated." While Microsoft has done excellent work, "they can't innovate and experiment as fast as a startup can."

Cursor Tab: Eliminating All "Low-Entropy" Actions

Arvid Lunnemark defines Cursor Tab's core goal as "eliminating all low-entropy actions inside the editor—when intent is already determined, it jumps you directly into the future."

• Mechanism Breakdown: Tab is not just autocomplete; it predicts the "next diff"—including editing code, jumping to different locations within a file, and even cross-file navigation. "The internal competition is: how many times can you make the user press Tab?"
• Technical Implementation:
• Sparse MoE Model: Input (code context) is large, output is tiny; the MoE architecture significantly improves long-context performance.
• Speculative Edits: Leverages the fact that "the original code itself is the strongest prior"—feeding the original code block back to the model, which mostly "agrees" and processes multiple lines in parallel, only generating tokens token-by-token at points of divergence. Result: diffs stream in, and users can start reviewing before generation finishes.
• KV Cache and Cache Warming: Pre-warms the cache for the current file's content as the user types; pressing Enter requires minimal prefill computation, significantly reducing first-token latency.
• Future Direction: Expanding from "predicting the next line" to "predicting the next 10 minutes"—including terminal commands, jumping to definitions to get type information and returning, etc. "Sometimes, what you'll do in the next 5 minutes can be predicted from what you just did."

Model Strategy: A "Hybrid Engine" of Custom Models + Frontier Models

Aman Sanger explains that Cursor achieves advantages in speed and quality through a hybrid architecture: "custom models handle specific tasks, while frontier models handle reasoning-intensive tasks."

• Cursor Tab Model: Custom-built for the "predict next edit" task, it even outperforms frontier models on internal evaluations.
• Apply Model: Solves the fatal weakness of frontier models when generating diffs—"they miscount line numbers." The frontier model outputs a rough code block, and a trained smaller model precisely applies it to the file. "You think this is a deterministic algorithm? Its failure rate exceeds 40%." At the same time, this strategy lets the frontier model output only a "sketch," drastically reducing token consumption (cost and latency).
• Model Routing: Exploring dynamic difficulty assessment to automatically select the right model. "Judging whether a problem is too hard for a level-4 model might itself require O1-level intelligence—that's hard."
• Early Assessment of O1: The team has integrated O1 but not included it in the default experience. "It doesn't stream output; you have to wait for a wall of text... It feels like a v0 version of test-time compute and search, with many things off." Sualeh Asif notes that O1 excels at reasoning problems (e.g., LeetCode) but "its ability to understand vague human intent is inferior to Sonnet."

The Future of Programming: Humans in the Driver's Seat, with Adjustable Abstraction Levels

Michael Truell emphasizes that the team's vision is "programmers in the driver's seat for the long term," maintaining control through adjustable abstraction levels, rather than reducing programming to natural language conversation.

• Against "Pure Natural Language Programming": Arvid Lunnemark uses the analogy of "pair programming"—sometimes saying "implement this function" is enough, other times "grabbing the keyboard and writing an example" is the most efficient communication. "The same applies to AI; sometimes showing an example is much faster than describing it in words."
• Core Principles: Speed + Control + Iteration Speed. "Much of the best engineering involves countless micro-decisions and difficult trade-offs... As long as humans are designing software, you want humans in the driver's seat."
• Future Form Speculation: Possibly like a "code browser with adjustable abstraction levels"—browsing a codebase in pseudocode, editing the pseudocode directly, with changes automatically mapped to the formal code layer. "You can point to any logical snippet and adjust the abstraction level up or down."
• Advice for Young Programmers: Aman Sanger believes "this is the most exciting time for programming"—programming in 2012 was full of boilerplate and tedious lookups; now "the fun density of programming has been massively increased." Arvid Lunnemark adds: "In the future, you can try first and then adjust, without spending five days standing in front of a whiteboard figuring out every detail—because the cost of trial and error has become extremely low."

Mentioned Positions

Position	Guest Attitude	Key Data
GitHub Copilot	Historical reference (early product, appears "stagnant")	Beta released in 2021; "no new features for a long time"
GPT-4 / GPT-4 Series	Key turning point (capability leap triggered Cursor's creation)	Early access obtained in late 2022
Claude Sonnet	Current "net best" coding model	"Superior to O1 in understanding vague intent"
OpenAI O1	Experimental/early stage (not included in default experience)	"Doesn't stream output, poor waiting experience"; "v0 version"
DeepSeek MLA	Technical reference (efficient attention mechanism)	"Compresses KV cache into latent vectors, then expands during inference"
LLaMA / Gemma	Technical reference (distillation/overtraining strategies)	Gemma 2B distilled from a 27B model

Judgments Worth Remembering

1. Michael Truell: "The best product 3-4 years from now will be far more useful than the best product today—the ceiling is extremely high, and stopping innovation means losing." — Support: Every year in AI coding, model capability leaps unlock new features; being a few months ahead creates a huge gap.

2. Arvid Lunnemark: "The goal of Tab is to eliminate all low-entropy actions—when intent is determined, jump you directly into the future." — Support: Speculative Edits leverage "the original code itself is the strongest prior," processing most code blocks in parallel.

3. Aman Sanger: "Frontier models can't even count line numbers correctly when generating diffs—you think this is a deterministic algorithm? Its failure rate exceeds 40%." — Support: Hence the training of a dedicated Apply model, letting the frontier model output only a rough sketch while the small model handles precise application.

4. Michael Truell: "Humans in the driver's seat—the best engineering involves countless micro-decisions and difficult trade-offs; you can't delegate all of that to a chat box." — Support: The team opposes the "pure natural language programming" vision, advocating for adjustable abstraction levels and retained control.

5. Sualeh Asif: "O1 excels at reasoning problems, but its ability to understand vague human intent is inferior to Sonnet." — Support: O1 performs well on in-distribution benchmarks, but "when you push it outside the benchmark distribution, Sonnet better maintains its capability."

6. Arvid Lunnemark: "In the future, you can try first and then adjust—without spending five days standing in front of a whiteboard figuring out every detail, because the cost of trial and error has become extremely low." — Support: Using codebase migration as an example, in the future you only need to show a few examples, and AI applies them everywhere automatically.

7. Michael Truell: "The third category of synthetic data—using verifiable systems (tests, formal proofs) to filter model outputs—is most likely to bring huge gains." — Support: Similar to the "monkey typewriter + Shakespeare verifier" paradigm, already proven effective in math and programming competition problems.

8. Aman Sanger: "We are not constrained by compute, but by ideas and engineering talent—even with $10 trillion, you can't quickly gather enough top-tier engineering talent." — Support: The engineering implementation from Transformer to GPT-3 (CUDA kernels, model parallelism) requires world-class engineers, which cannot be simply replicated with capital.