← Back to list
Colossus (Invest Like the Best / Business Breakdowns)Podcast1 Nov 2023Source: joincolossus.comHost: Colossus

Databricks: Data Based Decisions - [Business Breakdowns, EP.134]

In plain words

This is about Databricks, a private data platform valued at $430 billion with 50% growth. The author says its real edge is an 'open platform' that lets customers use any analytics tool or AI model without being locked in. Key holdings: Databricks (bullish, 85% gross margin), Snowflake (caution, weak in AI), and Cloudera (outdated, based on old Hadoop tech).

AI SummaryAI-generated · may contain errors · verify against the original

Databricks is a private company founded in 2013, originating from a UC Berkeley lab, with a latest valuation of $43 billion and still maintaining 50% growth as of summer 2023. Its core business involves providing data ingestion, transformation, and analysis tools to handle large-scale, multi-source,

~12 min full read · 9 sections
Deep Analysis

Databricks: Data Based Decisions - [Business Breakdowns, EP.134]

At a Glance

Yanev Suissa (Managing Partner at SineWave Ventures) breaks down the private company Databricks — a data processing platform born from a UC Berkeley lab, with a latest valuation of $43 billion and still maintaining 50% growth as of summer 2023. Suissa argues that Databricks' true moat is not the technology itself, but its "open platform" strategy — allowing customers to use all major analytics tools and AI models on the same platform, thereby avoiding lock-in. This fundamentally contrasts with closed platforms like Snowflake and Oracle.


1. Core Business: From "Structured vs. Unstructured" to "Lakehouse"

Suissa believes that the key to understanding Databricks lies in distinguishing between the two data types, and how the company started with unstructured data and eventually covered all scenarios.

  • Structured Data: Stored in predictable formats (databases, spreadsheets), such as a university course schedule (course name, professor, time, location). Snowflake started here.
  • Unstructured Data: Continuous data streams in rapidly changing environments (social media text/images/videos, sensor data). Databricks started here.

Real-World Application Scenarios:

  • Manufacturing: Customers like Honeywell collect real-time input weights, temperatures, sensor data, and video surveillance on complex production lines to adjust process parameters in real time.
  • Finance: Banks like Bank of America and JPMorgan detect fraud in real time—"It's useless to know fraud happened after the fact; it must be intercepted as it occurs."
  • Recommendation Systems: E-commerce platforms like Amazon analyze user behavior in real time to push ads at the optimal moment, for which advertisers pay a higher price.

Historical Comparison: Before Databricks, companies could only organize data (place it in files/databases) and analyze it afterward—"Everything was based on historical data." Real-time analytics capability is the fundamental revolution brought by Databricks.

Lakehouse: Databricks uses Delta technology to integrate data lakes (unstructured) with data warehouses (structured), generating over $200 million in revenue from zero within 18 months.


2. Competitive Landscape: The "Convergence-Divergence" Trajectory with Snowflake

Suissa argues that Databricks and Snowflake have undergone a three-phase evolution: "different starting points → product convergence → divergence again due to AI."

Dimension Databricks Snowflake
Starting Point Unstructured, real-time streaming data Structured, historical data
Current Core Differentiator Real-time analytics, AI-native capabilities Structured data warehousing
Gross Margin 85% ~56% (some revenue incurs AWS hardware costs)
Valuation Multiple ~19x (comparable to Snowflake) 19x
Million-Dollar Clients Over 300 Not disclosed

Suissa assesses: "AI fundamentally requires unstructured data analysis, which is Databricks' strongest muscle and the strongest in the industry—Snowflake does not come from this world. Going forward, you will see them increasingly resemble two independent entities: Snowflake becomes underlying infrastructure (similar to AWS/Microsoft), while enterprises that truly need real-time decision-making will invest in Databricks."

Relationship with Cloud Giants: Databricks and Snowflake pose a greater threat to Microsoft/Amazon than to each other—they enable clients to use cloud resources more efficiently (meaning they use less). However, Suissa believes "efficiency gains will be compensated by market expansion," and clients will always need underlying cloud infrastructure, so the three are "more symbiotic than competitive."


3. Open Source Strategy: The Unique Path from "Free" to "Profit-Making"

Suissa points out that Databricks has achieved "what almost no other company has done"—building a proprietary version on top of an open-source foundation, making customers feel that "the software is not only free but also equivalent to making money."

  • Origin: Apache Spark was born in the UC Berkeley lab in 2009 (open source), and Databricks was commercialized and founded in 2013. All 7-8 founders remain in senior management positions at the company.
  • Proprietary Strategy: Databricks built a proprietary version of Spark, allowing enterprises to use Spark's capabilities in a controlled environment. Open source itself has no competitive advantage, but Databricks leverages the open-source community for growth while differentiating through performance optimization (faster, lower cost).
  • Key Data: At the time of early investment, Databricks had a growth rate of 149%, and 40% of customers exceeded their memory quotas—indicating extremely high product value, with customers willing to overuse it.
  • "Cost-Saving Effect": Databricks significantly reduced underlying hardware (cloud service) costs, to the point that "the software is not only free but also equivalent to making money."

Suissa reminds readers to consider the investor's perspective: The author uses the "open source → proprietary → cost-saving" argument to demonstrate Databricks' unique advantage, but this is from an investor's viewpoint—companies like Red Hat have attempted similar paths, with very few succeeding.


4. Moat and Risk: Stickiness vs. AI-Driven "Forced Switching"

Suissa argues that the switching costs for data platforms are extremely high, but AI may serve as a "forcing function" to break existing stickiness.

Sources of Stickiness:

  • Once customers migrate their data, developers, and workloads to a platform, the switching costs are enormous — "this is a cross-generational decision."
  • Databricks recognized this early on and aggressively hired a sales team, "scaring the people at Snowflake."

How AI Changes the Game:

  • "Once AI truly matures on the enterprise side, companies must use it or lose their competitive edge. And Databricks enables that — I don't see any other company keeping up in the necessary way."
  • Databricks' AI advantages: Security (meeting financial/healthcare/public sector requirements), Data Governance (controlling proprietary data from leaking), Model Neutrality (allowing use of any model, including self-built ones).
  • After acquiring Mosaic, it launched its third-largest product line: running AI models and integrating them with enterprise proprietary data.

Risks:

1. Maintaining an open platform ethos: It must not become a closed system, or it will be "Hadoop'd" by the next wave of innovators.

2. Sustaining leadership in security and governance: AI tools themselves lack these capabilities, and Databricks must continuously provide them.

3. IPO timing: Suissa believes "Databricks will go public when investors are hungry for growth again" — implying the current market environment is unfavorable.

Meaning of "Being Hadoop'd": Cloudera (based on Hadoop) was once the largest data processing framework, but Hadoop used batch processing (processing one item at a time), while Databricks invented in-memory processing (scanning the entire bookshelf simultaneously). Speed, flexibility, cost advantages, and real-time capabilities allowed Databricks to replace Cloudera.


5. Growth Engines: Verticalization and Partner Ecosystem

Suissa shared how SineWave helps Databricks accelerate growth—a typical narrative from the holder's perspective, but the specific strategies are worth noting.

Partner Channel:

  • Deloitte: Trains its consultants so that all Deloitte clients can deploy Databricks—"significantly expands sales capacity for smaller companies"
  • IBM: Similar collaboration
  • Platform openness: Allows other analytics tools to operate on the same platform, becoming a "one-stop shop"—analogous to Kayak (aggregating all airlines)

Verticalization Strategy:

  • Early stage: Technology (Spark) → Product (Databricks proprietary version) → Solutions (targeting specific industries)
  • Began focusing on vertical industries about 6 years ago: Healthcare, Industrial, Public Sector (SineWave helped build the public sector business from scratch, now "hundreds of millions of dollars, highly sticky")
  • Accelerates adoption by customizing so clients "understand Databricks in their own context"

Revenue Model:

  • Pay-as-you-go model billed by the second
  • Fortune 500 client annual spending: Seven figures to double-digit millions of dollars
  • If AI takes off fully, a single client could contribute "double-digit millions of dollars"

Mentioned Positions

Position Guest Stance Key Data
Databricks Bullish (Holding) Valuation $43 billion, 50% growth, 85% gross margin, over 300 million-dollar clients, 149% early-stage growth rate
Snowflake Risk Warning (Competitive but Differentiated) 56% gross margin, 19x valuation multiple, strong in structured data but weak in AI capabilities
Cloudera Risk Warning (Already Disrupted) Hadoop-based batch processing, replaced by Databricks
Microsoft Azure Neutral (Symbiosis/Competition) Key partner, higher security rating than AWS, helps open enterprise market
AWS Neutral (Symbiosis/Competition) Early sole cloud platform, later surpassed by Microsoft
Honeywell Neutral (Client Case) Real-time manufacturing data analytics
Bank of America / JPMorgan Neutral (Client Case) Real-time fraud detection
Deloitte Neutral (Partner) Amplified channel sales capabilities
IBM Neutral (Partner) Similar channel partnership to Deloitte
OpenAI Risk Warning (Competition) Models not open for commercial use, usage restrictions
Oracle Risk Warning (Negative Case) Closed platform leads to client churn
Apple Neutral (Analogy) Closed platform being broken by regulation

Judgments Worth Remembering

1. "Databricks and Snowflake pose a greater threat to Microsoft/Amazon than to each other" (Suissa) — They enable customers to use cloud resources more efficiently, but the efficiency loss is offset by market expansion, and customers will always need the underlying cloud.

2. "AI will break the sticky network effects of data platforms" (Suissa) — Even if customers are locked into Snowflake, they must switch to Databricks to use AI, because "real-time analytics capabilities, governance tools, and security integrations far surpass those of other companies."

3. "Databricks has achieved what almost no other company has — building a proprietary version on top of open source, making customers feel the software is not only free but essentially generating profit" (Suissa) — This is achieved by significantly reducing underlying hardware costs.

4. "Being Hadoop'd means being replaced by faster technology — Hadoop's batch processing vs Databricks' in-memory processing is like organizing books one by one vs scanning an entire shelf simultaneously" (Suissa) — Speed, flexibility, and real-time capabilities are the keys to disruption.

5. "The journey from technology to product to solution is one every company must take" (Suissa) — Databricks initially focused on technology (Spark), then moved to products (proprietary version), and six years ago began verticalizing solutions (healthcare/industrial/public sector), accelerating growth.

6. "Open platform vs closed platform: Oracle and Apple make customers adjust their needs to fit the platform, while Databricks says, 'No, you don't need to adjust, because we have already built in all adjustments and options'" (Suissa) — Suissa believes that in the long run, most companies will become marketplace platforms, and closed platforms will eventually be broken by regulation or market forces.

7. "The timing of the IPO depends on when investors once again crave growth — Databricks' growth stream, customer depth, gross margins, and growth rates have never changed; it's just that market sentiment is absent" (Suissa) — This implies the current valuation environment is unfavorable, but fundamentals remain strong.

8. "Dolly (Databricks' proprietary LLM) is not the only option — you can use any model, including self-built models" (Suissa) — The model-agnostic strategy allows Databricks to avoid betting on a single AI winner and instead become the "enterprise-grade security layer" for all AI models.