#398 – Mark Zuckerberg: First Interview in the Metaverse

This Issue at a Glance

Mark Zuckerberg (Meta CEO) and Lex Fridman engaged in a conversation in the metaverse using realistic Codec Avatars, exploring how virtual spaces can enable deep human connection over long distances through 3D spatial audio and facial expression capture technology. The core thesis is that this technology is key to future interpersonal interaction on the internet, capable of conveying emotional nuance. Key conclusions include: Meta is developing AI personalities (such as a Snoop Dogg-style character), but the technical challenges are immense, requiring the encapsulation of human personality quirks and aesthetics. The conversation emphasizes that while physical spaces may be hundreds of miles apart, virtual spaces can create a sense of being in the same room, driving a transformation in social experiences.

Theme 1: Codec Avatars — "Real" Connection Across Physical Distance

Zuckerberg believes the core value of Codec Avatars lies in conveying "presence" — giving remote interactions the same psychological and social benefits as physical co-location.

• Technical Principle: Users undergo professional scanning (currently taking several hours) to generate a computer model of their face and body, which is compressed into a codec; the headset captures facial expressions in real-time, transmitting the encoded avatar to the other party. Zuckerberg notes this is more bandwidth-efficient than transmitting full video or 3D immersive video.
• Beyond the "Uncanny Valley": Fridman emphasizes that facial details (freckles, wrinkles, subtle eye movements) are key to immersion, stating, "Perfection is not the source of immersion; it's those subtle imperfections." Zuckerberg adds that millimeter-level movements of the eyes and eyebrows can change emotional expression, something traditional cartoon avatars cannot achieve.
• Future Path to Adoption: Current scanning takes hours, but Meta's goal is to allow users to complete a scan with their phone in 2-3 minutes, generating an avatar of equivalent quality. Zuckerberg expects a gradual rollout over "the next few years," starting with work scenarios (e.g., Workrooms) and expanding to social and gaming.

Theme 2: Mixed Reality — The Fusion of Digital and Physical Worlds

Zuckerberg defines the Quest 3 as "the first mainstream mixed reality device," believing it will usher in a new paradigm where digital objects are overlaid onto the physical environment.

• Hardware Breakthrough: The Quest 3 is priced at $500, two-thirds cheaper than the Quest Pro ($1,500), but offers superior mixed reality performance — several times higher resolution, a custom next-generation chipset developed with Qualcomm, and is 40% thinner. Zuckerberg emphasizes that each new headset generation brings "major new capabilities," unlike the minor iterations of smartphones.
• Use Cases: Mixed reality allows users to be immersed in 3D experiences while still seeing the physical environment (e.g., furniture, other people), avoiding collisions with obstacles. Fridman gives an example: in a shooting game, you can hide behind a real sofa to avoid fire. Zuckerberg envisions a future where the number of holograms in a room rivals physical objects, stating, "Any screen can be a hologram, and any game (like table tennis) is more convenient as a hologram."
• Social Impact: Zuckerberg argues that the digital world has become too large to be confined to small screens, while the physical world is too important to sit still all day. Mixed reality is "the most human, most social computing platform," allowing people to move their bodies while enjoying digital convenience.

Theme 3: AI Personalities — From Tools to Social Companions

Zuckerberg proposes that the future will not have a single super AI, but rather diverse AI personalities serving entertainment, commerce, and social connection.

• Product Matrix: Meta is building three categories of AI:
• General Assistant (Meta AI): Faceless, text-based, providing factual assistance.
• Business AI: Helps small businesses handle customer inquiries (e.g., bookings, after-sales) without requiring human staff.
• Entertainment AI: Such as a Snoop Dogg-style "Dungeon Master" that can join group chats to tell jokes or play text-based adventure games. Zuckerberg reveals that the most popular internal test was the "advanced text-adventure Dungeon Master."
• Technical Challenges: Creating a brand-new character is relatively simple (e.g., Snoop Dogg playing a Dungeon Master), but making an AI accurately represent a real creator (e.g., a Lex Fridman AI doppelganger) is far more difficult. Zuckerberg points out that the key is not avatar expression, but making the AI model "reliable and predictable," ensuring it doesn't say things against the creator's wishes. This will require testing starting "next year."
• Open Platform: Meta is developing "AI Studio," allowing anyone to create their own AI, similar to creating UGC content. Zuckerberg believes community creativity will unlock the full potential, but this needs to be rolled out in stages.

Theme 4: Ethics and Identity — Philosophical Questions of Digital Existence

The conversation touches on digital identity, death, and the ethical boundaries of AI, with Zuckerberg acknowledging these are "issues that society needs to explore together."

• Identity Fluidity: Zuckerberg notes that Codec Avatars can present a "best version" of oneself (e.g., shorter hair, cleaner beard), raising the philosophical question of "who we are" — are we the entity driving the avatar, or the avatar itself? He analogizes that physical bodies also fluctuate, but digital versions may be more persistent.
• Digitizing the Deceased: Fridman raises the possibility of conversing with deceased loved ones. Zuckerberg responds that Meta already has experience handling digital content of deceased users (e.g., memorial accounts), but "needs to study the balance" — helping with mourning while avoiding unhealthy dependency. He also notes that AI doppelgangers could be created while the user is alive, helping creators interact with their community, which is "not a post-mortem thing."
• Virtual Behavior Norms: Zuckerberg believes the definition of "harm" differs in the digital world — shooting a friend in a game is fun, but anonymity can lead to toxic behavior. He advocates for "as much freedom as possible, unless it harms the rights of others," but specific rules will need to be gradually figured out as technology evolves.

Mentioned Positions

Position	Guest Sentiment	Key Data
Meta (META)	Bullish (core business direction)	Quest 3 priced at $500, 66% cheaper than Quest Pro; Llama 2 downloads exceeded expectations
Qualcomm (QCOM)	Neutral (partner)	Provides custom chipset for Quest 3
Snoop Dogg (Private)	Neutral (AI personality partner)	Plays the role of AI Dungeon Master

Judgments Worth Remembering

1. Zuckerberg: Codec Avatars are more efficient than video transmission — By encoding a facial model rather than transmitting full video, bandwidth requirements are lower, while still capturing key immersion details like "freckles, wrinkles, and subtle eye movements."

2. Zuckerberg: Mixed reality is "the most human computing platform" — It allows users to move their bodies (the importance of the physical world) while enjoying digital convenience, unlike traditional computers that require "sitting still all day."

3. Zuckerberg: The future will not have a single super AI, but diverse AI personalities — From general assistants to a Snoop Dogg-style Dungeon Master, AI will serve entertainment, commerce, and social connection; Meta is building "AI Studio" for users to create their own.

4. Fridman: Digital existence may be more "real" than physical existence — "Realness" depends on subjective experience rather than spatiotemporal physics; Codec Avatars give remote interactions the same psychological and social benefits as co-location.

5. Zuckerberg: AI doppelgangers are harder to create than brand-new characters — Making an AI accurately represent a real creator requires ensuring it is "reliable and predictable," not saying things against their wishes; this is far more complex than creating a character from scratch and is expected to be tested next year.

6. Zuckerberg: Digital identity raises the philosophical question of "who we are" — Avatars can present a "best version" of oneself (e.g., shorter hair), challenging the uniqueness of physical identity, requiring study by social scientists and psychologists.

7. Zuckerberg: Meta will roll out Codec Avatars in stages — Starting with work scenarios (e.g., Workrooms), then expanding to social and gaming, with the ultimate goal of allowing users to complete a scan with their phone in 2-3 minutes.

8. Zuckerberg: The definition of "harm" differs in the digital world — Shooting a friend in a game is fun, but anonymity can lead to toxic behavior; rules should be "as free as possible, unless harming the rights of others," but will need to be gradually figured out as technology evolves.