This interview covers how DeepMind scientist Oriol Vinyals used AI to play StarCraft. He says the biggest challenge isn't speed but 'exploration'—with so many possible moves, random trial-and-error fails, so AI must first learn by watching human replays. Key highlights: AlphaStar (DeepMind's AI) beat pro player MaNa after combining imitation learning and self-play. Also mentioned: Blizzard (Activision Blizzard), which provided public game data.
DeepMind Senior Research Scientist Oriol Vinyals discussed the AlphaStar project he led on the Lex Fridman podcast, an AI agent that defeated top professional players in StarCraft. Vinyals' research has been cited over 39,000 times, with core contributions including sequence-to-sequence learning, au
Oriol Vinyals is a senior research scientist at Google DeepMind with over 39,000 citations, and a core contributor to sequence-to-sequence learning, neural machine translation, and the AlphaStar project. In this episode, he and Lex Fridman delve into the technical path behind AlphaStar defeating professional StarCraft players, as well as the relationship between deep learning, sequence modeling, and general intelligence.
The most impactful insight in the entire episode: Vinyals argues that the greatest challenge in StarCraft is not partial observability or real-time constraints, but the "exploration problem" — because the action space is so vast, random exploration makes it nearly impossible to naturally discover any effective strategy, and human demonstrations (imitation learning) must be used to guide the process.
Vinyals has been deeply involved with StarCraft since 1998, preferring the Random race to fully understand the game’s mechanics. He believes that by playing all races personally, one can gain a deeper understanding of the opponent’s "pain points"—"If you understand a race because you play it, and you also know what is annoying, then when you are on the other side, you know what to do to annoy that person." His best race is Zerg, but he often chooses Random in matches to accumulate comprehensive experience.
Vinyals likens StarCraft to "partially observable chess with an economy": Players start from scratch, needing to gather resources (minerals and vespene gas), balance economy and military strength, scout and counter-scout without seeing the opponent, and make all decisions continuously in real time. He recalls that when StarCraft was released in 1998, the internet in Spain was still underdeveloped; the game was initially an offline experience, and it was only through LAN and Battle.net that the world of online competition truly opened up.
Vinyals believes the social impact of games like StarCraft and World of Warcraft is underestimated: They allow players to connect with people they would not encounter in real life, understanding the diversity of the world through virtual realms. He still maintains Facebook friendships with players he met online back then.
Vinyals believes AlphaStar’s core breakthrough lies in using imitation learning to solve the “exploration problem” in StarCraft. He explains that if reinforcement learning relies solely on random actions, it is nearly impossible to naturally discover effective strategies such as “building expansion bases”—because any early random action would pull workers away from mining, causing an economic collapse. Therefore, an “imitation policy” must first be trained using human replay data (Blizzard’s public dataset contains a large number of high-MMR player matches).
Key technique: Vinyals’ team informed the neural network “what level of player it is imitating” during imitation learning. They trained on replay data from 3,500 MMR and above, but labeled each replay with a specific MMR score. After training, a “level parameter” could be set to run the policy at different skill levels—when set to 3,000 MMR, the policy had weaker economies and lower APM (actions per minute); when set to 6,000 MMR, the policy could defeat all built-in AIs, but still fell far short of actual 6,000 MMR players.
From imitation to surpassing: AlphaStar League was the key. Vinyals points out that pure self-play tends to fall into local optima and cannot cover all strategy types. AlphaStar League created an “agent ecosystem” where different agents developed distinct “personalities”—some pursued “cheese” (extreme all-in strategies), some followed economic routes, and others adopted aggressive styles. Newer generations of agents had to compete against all previous ones, thereby learning to counter various strategies. Vinyals likened this to “creating a Battle.net for agents.”
At the architecture level, Vinyals emphasized that AlphaStar’s neural network is highly similar to language models: It uses Transformer and LSTM to process time series, encoding game observations (spatial map + unit list) into sequences, and then predicting the next action. He stated bluntly: “This is almost the same code as machine translation, except the input is observations instead of words, and the output is actions instead of translations.”
Vinyals admitted he was initially a "pessimist," predicting AlphaStar would lose 0-5 to TLO. Within the team, there was a bet, and he sent an email predicting a crushing defeat. After AlphaStar defeated TLO (a professional player, though Protoss was his off-race) 5-0, the team celebrated at a bar. David Silver then proposed inviting a stronger Protoss specialist, MaNa (with an MMR roughly 1,000 points higher). Vinyals thought there was a "99% chance of losing," but AlphaStar won again.
Vinyals emphasized that this does not mean AlphaStar is already "superhuman" or "invincible." He clearly stated: "These agents are at the Grandmaster level, but they are not invincible, nor are they perfect." The existence of cheese strategies in StarCraft means there is always a possibility of winning through high-risk maneuvers. He observed that AlphaStar still has notable weaknesses, such as insufficient scouting and poor responses to cloaked units.
Vinyals found MaNa's reaction after losing particularly meaningful: "He had no excuses... He was simply defeated. Watching a human being surpassed by an AI system is a beautiful moment for a researcher."
Vinyals believes the core challenge of deep learning is "generalization." He points out that current neural networks essentially fit data distributions and fail when encountering out-of-distribution samples (e.g., adversarial examples). He distinguishes two paths: one is the "scale path" (more data, larger models), which has proven effective in StarCraft and machine translation; the other is the "discretization plus program synthesis path," which combines neural networks with deterministic programs to achieve stronger generalization guarantees—"If you write a sorting code, you can prove it works for all possible inputs."
Vinyals is cautious about the "Turing test": He believes fully passing the Turing test is "still premature," because "statistical methods may not be enough; we need a breakthrough." Unlike StarCraft, he cannot outline a clear execution plan for the Turing test. However, he acknowledges that progress can sometimes be unpredictable—"Six months ago, I couldn't have predicted that AlphaStar would reach Grandmaster level."
Vinyals views "meta-learning" (learning to learn) as a key step toward AGI: His ideal system should be able to absorb new tasks without retraining from scratch—"If I train a Protoss vs. Protoss network and want to play Terran vs. Terran, I have to discard those weights and retrain. If I see a method that can absorb new problems without restarting the process, that would be a good definition of AGI for me."
On AI safety, Vinyals expresses no short-term concerns but advocates long-term vigilance: "I'm more afraid of other planetary-scale threats... But we should prepare for the potential misuse of technology, just like any other technology."
| Position | Guest Sentiment | Key Data |
|---|---|---|
| AlphaStar (DeepMind) | Bullish (technological breakthrough) | Defeated 7,000 MMR professional player MaNa; APM limited to 300-800 (professional human level); Uses Transformer + LSTM architecture |
| Blizzard (Activision Blizzard) | Neutral (partner) | Provided public replay dataset; StarCraft released since 1998 |
1. Vinyals believes the greatest challenge in StarCraft is the "exploration problem" — random actions almost never naturally discover any effective strategy, requiring human demonstrations for guidance. Support: Any early random action will pull mining workers away, leading to economic collapse.
2. The "skill-level labeling" technique in imitation learning is a key innovation of AlphaStar — it tells the neural network what MMR-level player it is imitating, thereby controlling the output level of the strategy. Support: When set to 3,000 MMR, the strategy deteriorates; when set to 6,000 MMR, it can defeat all built-in AI.
3. AlphaStar League is "a Battle.net created for agents" — by maintaining a diverse population of agents (cheese type, economic type, aggressive type), new generations of agents must learn to handle all strategies, avoiding local optima from pure self-play.
4. Vinyals distinguishes two paths for deep learning: the scaling path vs. the discretization + program synthesis path — the former (more data, larger models) has proven effective but is "a bit boring"; the latter (combining neural networks with deterministic programs) offers stronger generalization guarantees but remains underdeveloped.
5. Vinyals holds a "cautiously pessimistic" view of the Turing test — unlike StarCraft, he cannot outline clear execution steps for the Turing test, believing that "statistical methods may not suffice; we need a breakthrough."
6. Vinyals regards "meta-learning" as a key definition of AGI — an ideal system should be able to absorb new tasks without retraining weights from scratch; he acknowledges that currently, "Protoss weights are completely useless for Terran."
7. Vinyals believes the social impact of StarCraft is underestimated — it allows players to connect with people they would not encounter in real life, understanding the diversity of the world through a virtual environment.
8. Vinyals' research methodology: "Find a hard problem, then see how research fits it" — machine translation drove sequence-to-sequence, combinatorial algorithms drove pointer networks, and StarCraft drove large-scale imitation learning and AlphaStar League.