Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

Below is the analysis and interpretation based on the interview transcript you provided.

At a Glance

Guests: Gavin Uberti and Rob Wachen, co-founders of AI chip company Etched.

Main Theme: The two young founders describe how they are challenging NVIDIA by building an AI chip and rack system optimized for inference through vertical integration and two core technical bets (low-voltage inference and cluster-scale memory), having already secured over $1 billion in customer demand.

Most Significant Judgment: Gavin Uberti believes that inference will become one of the largest markets in the world, and the company that produces the most tokens will become the most valuable company in the world. This judgment drives all of Etched's decisions regarding product design, vertical integration, and capacity expansion.

Key Topics

1. Two Technical Bets: Low-Voltage Inference and Cluster-Scale Memory

Gavin Uberti argues that existing GPU architectures are designed for training, not inference. Etched achieves fundamental performance breakthroughs through two key technical bets.

• Mechanism Breakdown: Inference is divided into two phases: "pre-fill" (loading model memory) and "decode" (generating output tokens). Etched has made core bets for each phase.
• Pre-fill (Low-Voltage Inference): The key is floating-point operations per second (FLOPS) and FLOPS density. However, increasing FLOPS leads to thermal throttling. Gavin points out that by lowering voltage (Denard scaling), power consumption decreases quadratically. Etched developed "low-voltage inference" technology, operating at less than half the voltage of any other AI chip, allowing them to pack more FLOPS into the same silicon area without thermal throttling.
• Decode (Cluster-Scale Memory): The key is memory bandwidth. Gavin believes people are asking the wrong question; instead of looking at single-chip bandwidth, one should look at "the memory bandwidth of the entire scaled cluster." By developing its own interconnect, Etched reduces chip-to-chip latency by more than 5x compared to NVIDIA Blackwell's ~4000 nanoseconds, enabling the memory (SRAM and HBM) of all chips in a cluster to be used as a unified memory pool, significantly boosting decode speed.
• Data Support: Gavin emphasizes that GPU model FLOPS utilization (MFU) is typically between 20-50% and cannot reach 100% due to thermal throttling. Etched's interconnect latency is "more than 5x lower" than NVIDIA's.

2. Production is the Product: Extreme Vertical Integration and Parallelization

Rob Wachen believes Etched's competitive advantage lies not only in chip design but also in the ability to perform extreme vertical integration and parallel development of the entire rack system (from chip to cold plate to manufacturing).

• Mechanism Breakdown: Etched does not just build a chip; it delivers a complete inference rack, including chips, circuit boards, power supplies, interconnects, and manufacturing. They believe "the best part is no part, the best supplier is no supplier," internalizing as many steps as possible.
• Data Chain and Historical Context: To accelerate development, Etched performed extensive "prefetching" work before chip tape-out:
• Manufactured "thermal chips" with the exact same thermal profile as the real chip to validate cold plates, ensuring no leaks upon the chip's arrival.
• Built its own factory in Taiwan and cloned a test station setup in its office.
• Deployed the rack (without chips) to customer data centers before the chips arrived, booting the software stack.
• Simulated the entire chip using over 700 FPGAs, running the inference software stack in advance.
• Result: Another well-known AI chip company took 10 months from receiving chips to running inference, while Etched took only 40 days.
• Deduction: This model allows Etched to achieve faster product iteration and time-to-market with far fewer people than large companies (e.g., NVIDIA's 20,000 employees).

3. Talent Philosophy: A Binary Combination of "Legends" and "People with Chips on Their Shoulders"

Gavin Uberti and Rob Wachen describe a unique hiring strategy: pairing industry legends with driven young geniuses (chips on shoulders) to solve the hardest technical problems.

• Mechanism Breakdown:
• Legends: For each hardest technical problem, they find the best person in the world through "project-based hiring" (e.g., Brian Loyler, founder of NVIDIA's HGX/DGX systems). They believe the capability gap between the first and tenth best is enormous.
• Young Geniuses: People like Gavin's high school robotics competition partner, Sanford, who "don't know where the bodies are buried," are willing to take huge risks and think from first principles.
• Collaboration Model: Legends provide experience and "billion-dollar lessons," while young geniuses provide speed and the courage to break conventions. The two must work closely together; neither is sufficient alone.
• Deduction: This model is self-selecting. Gavin notes that only someone "with a screw loose" would join Etched when it looks so unproven on paper. As the company succeeds, they worry about losing this "contrarian" culture and must deliberately maintain it.

4. The Darkest Hour of Fundraising and the "Assume It's Feasible" Mindset

Rob Wachen and Gavin Uberti share the company's difficult fundraising period in early 2024, emphasizing that the core mindset for overcoming all obstacles is "assuming the problem is solvable."

• Historical Context: In early 2024, Etched completed its architecture design and was preparing for the physical design phase. They realized they needed to build not just a chip but an entire cluster, requiring far more than the $15 million they had in the bank. They estimated needing $100 million, but the record for an AI chip Series A round at the time was around $40-50 million.
• Data Chain: They spent 100 hours writing a 30-page memo, but almost all major venture capital firms passed. Forced into "survival mode," they started with small investments ($1 million, $2 million) and eventually cobbled together a $103 million Series A round through a combination of debt and equity.
• Mechanism Breakdown: Faced with seemingly unsolvable technical problems (e.g., aligning two clock signals within 50 picoseconds), Gavin's philosophy is: "Assume the problem is solvable, then figure out how to solve it. " They solved this challenge with an ingenious phase drift mechanism, even though a team member resigned over it, believing it impossible. This "find a way mentality" is core to the company culture.

Mentioned Positions

Position	Guest Attitude	Key Data
NVIDIA	Risk/Comparison Target	Chip-to-chip latency ~4000 ns; Blackwell B300 FLOPS density lower than some competitors; HGX/DGX systems contribute majority of revenue.
TSMC	Bullish/Key Partner	Customer service "far exceeds any other industry"; willing to run experiments at its own expense to improve Etched's yield.
Synopsys	Bullish/Key Partner	Provided simulators on extremely favorable terms when Etched was cash-strapped (equivalent to a large loan).
Google (TPU)	Risk/Comparison Target	Guest believes TPU is not "existential" for Google, as its core revenue comes from search.
Meta (MTIA)	Risk/Comparison Target	Same as above; MTIA is not "existential" for Meta.
Microsoft (Maya)	Risk/Comparison Target	Same as above; Maya is not "existential" for Microsoft.
OpenAI (Jalapeño)	Risk/Comparison Target	Same as above; Jalapeño is not "existential" for OpenAI.
Cursor	Bullish/Case Study	Used multiple coding agents to build a complete browser from scratch in one week.
Exnor	Background Mention	Gavin's first job; later acquired by Apple for $200 million.
Octo	Background Mention	Company where Gavin worked; later acquired by NVIDIA for several hundred million dollars.

Judgments Worth Remembering

1. Inference will become one of the largest markets in the world, and the company that produces the most tokens will become the most valuable company. (Gavin Uberti) — This drives all of Etched's decisions on how to bring the most token capacity online as fast as possible.

2. "The best part is no part, the best supplier is no supplier." (Rob Wachen) — Extreme vertical integration (chip, board, cold plate, interconnect, manufacturing) is key to achieving performance and speed.

3. "Assume the problem is solvable, then figure out how to solve it." (Gavin Uberti) — Facing "impossible" problems like 50-picosecond clock alignment, this mindset is core to overcoming fear and finding solutions.

4. "Production is the product." (Rob Wachen) — Preparing the rack, software, cold plate, and production line before chip tape-out allowed Etched to go from receiving chips to running inference in just 40 days, compared to another company's 10 months.

5. "Machines don't think like people. For people, storing and loading memory is cheap, doing math is expensive; for chips, it's the opposite." (Gavin Uberti) — This explains why future models should be designed to use massive computation (e.g., activating more experts, processing ultra-long contexts) rather than relying on efficient memory access.

6. "For our product, success is existential; for Google, Meta, Microsoft, OpenAI, the failure of their chip projects is not existential." (Newly hired Etched architect) — This explains why a startup focused on a single product can attract top talent and build the best chips.

7. "We are in an era where the cost of 'intelligence' is far lower than the value of 'intelligence,' meaning we will face a multi-year or even multi-decade shortage of token supply." (Rob Wachen) — Any chip or system that can produce tokens will be extremely valuable.

8. "By 2027, the number of 'agents' doing knowledge work will exceed the number of humans." (Rob Wachen) — This suggests that the future metric for measuring societal productivity will shift from "GDP per capita" to "agents per megawatt."