Why Managed Platforms Won

The shift you are living through

Ten years ago a data team ran its own Hadoop / on-prem MPP cluster: fixed nodes, a sizing committee, a 3am pager when a disk filled. Today most analytics runs on a managed cloud platform you never SSH into. Three forces drove that:

• Elasticity — On a fixed cluster, the month-end report and the idle Sunday cost the same. Managed platforms let compute scale to zero when nobody queries, and burst for the heavy job — you pay for work done, not hardware owned.
• Separation of storage and compute — The single biggest architectural idea in this track. On Hadoop, data lived on the compute nodes (HDFS), so to store more you bought more CPU you didn't need (and vice-versa). Cloud platforms put data in cheap object storage (S3 / ADLS / GCS) and spin compute up next to it on demand. Storage and compute now scale — and bill — independently.
• Operability — No cluster to patch, no rebalancing, no JVM tuning. The platform owns uptime; you own SQL and cost.

Use Case Example: A retailer's BI dashboards are idle overnight but hammered 9am–6pm. On a fixed cluster they paid for peak 24/7. On Snowflake/BigQuery/Databricks, compute auto-suspends at night and auto-scales at the open — same workload, a fraction of the bill — while the data sits untouched in object storage either way.

Self-hosted clusters are like owning a generator for your house: you buy it for peak load (the one hot day with the AC maxed), then pay to idle it the other 360 days. A managed platform is the electricity grid: the power plant (storage) and your meter (compute) are separate, you draw exactly what you use this instant, and someone else owns the 3am maintenance. You stopped buying generators not because generators are bad, but because metered, elastic supply is cheaper for spiky demand — which analytics always is.

Storage and compute, decoupled

One shared, durable storage layer in object storage; many independent, ephemeral compute engines reading from it. Add a warehouse/cluster/slot pool without copying a single byte of data — and turn it off without losing any.